When you import a Python package for use in your code, it searches folders of your computer in a particular order to find the package you are asking it to run.

This order is defined in your Python’s sys.path variable. Unless you manually changed it, the order is:

1. System packages: those that already come built-in whenever you installed Python
Location of my system packages folder:

/Users/nweimer/anaconda3/lib/python3.6/

2. Third Party Packages aka “Site Packages”: those that you installed yourself using a package manager such as pip (“pip install packagename”) or conda (“conda install packagename”)

Location of my site packages folder:

/Users/nweimer/anaconda3/lib/python3.6/site-packages/

We can view this order by the Python code below:

import sys
print('\n'.join(sys.path))

Python uses the first instance of the package that it finds. So, for example, if we search for some package XYZ and XYZ is in both system packages and site packages folders, then Python will use the package XYZ as defined in the system packages directory (assuming the system packages directory comes before your site packages directory in sys.path).

Using a homemade Python package

Your package must either be: 1) physically located in one of the paths listed in your sys.path variable (typically your Site Packages directory) or 2) symbolically/hard linked to one of the paths listed in your sys.path variable.

Side note: What is hard linking vs symbolic linking? Essentially, hard linking a file creates an exact replica of the linked file. This means you can accidentally delete the original file and still not lose your work- the hard link will still exist. A symbolic link simply points to the name of the linked file to mirror what’s in the file. If you accidentally delete the original file, you are out of luck and would lose your data because the symbolic link would now point to a file that does not exist. If interested to learn more, check out this Stack Overflow discussion.

Below we will use symbolic linking. Symbolic linking a file to another directory allows that file to be accessed from the other directory in addition to the directory where the file actually lives.

To symbolically link a file on Mac, use the following:

ln -s /path/to/original/file  /path/to/linkName

Note: If you’d rather hard link, use the above command without the “-s” option.

Now that we know how to link files, we now can simply write our own Python packages and link them to the site packages Python path! This way, whenever you go to import your Python package into your code, Python can actually have a copy of your package ready to go!

An example of symbolic linking on my computer (I created a package called dataset_explorer):

ln -s /Users/nweimer/Desktop/python_modules/dataset_explorer.py /Users/nweimer/anaconda3/lib/python3.6/site-packages/dataset_explorer.py

You can find the contents of this dataset_explorer.py at the following GitHub link.

The last thing we need to do is to import the package into a new script, which is the same syntax as if you were importing any site package or third party package:

from dataset_explorer import *

Now that you know how to import your own Python packages, you should learn about a special function that many people use at the top of their homemade packages called __main__. You can learn more about this special function at this YouTube video.

That’s it! You’re free to use the functions describe_vars() and frequency_encoding() defined in this homemade package (dataset_explorer.py) in your Python code!

References:

https://leemendelowitz.github.io/blog/how-does-python-find-packages.html

https://realpython.com/python-virtual-environments-a-primer/

https://docs.python.org/3/library/sys.html#sys.path

https://docs.python.org/3/tutorial/modules.html#packages

https://stackoverflow.com/questions/3387695/add-to-python-path-mac-os-x