1. Introduction
Contents
1. Introduction#
Notes#
Jupyter notebooks are a fantastic tool: you can write some code and get immediate feedback. It makes data science a lot more enjoyable!
Yet, they have problems: they can get hard to manage, reproduce and test.
That’s why many teams only allow their use for prototyping but do not use it in production.
This course will teach the practices and tools to use Jupyter notebooks as a tool for production work effectively.
At Ploomber, we’ve helped dozens of companies deploy notebooks into production: from small startups to Fortune 100 companies, so you’re in good hands.
Fun fact: this course is entirely developed from Jupyter!
What you’ll learn?#
Write clean Jupyter notebooks
Testing (unit, integration)
Build data pipelines from Jupyter
Cloud deployment
Preparing your environment#
To ensure you can run all the examples and get all the packages we’ll use, we highly recommend installing miniconda.
Ensure you’re able to create a conda environment:
# create environment
conda create --name some-env python=3.10 --yes
# activate it
conda activate some-env
You can also use other tools such as venv, but some packages might be harder to install. Whatever tool you choose, feel free to request help in the #notebooks-academy channel in our Slack.
Commands used in the lesson#
Here are the commands that we used during the video lesson:
# get the code
git clone https://github.com/ploomber/notebooks-academy
# move to the folder with the code
cd notebooks-academy/website/lessons/01/code
Convert .ipynb to .py:
pip install jupytext
jupytext notebook.ipynb --to py:percent
Run unit tests:
pip install pytest
pytest test_notebook.py
Generate data pipeline:
pip install soorgeon
mkdir pipeline
cd pipeline
soorgeon refactor ../notebook.ipynb
Get involved!#
Join our community, and ask us anything in the #notebooks-academy channel. We’ll be answering questions and getting feedback over there. Help us make this online course a success!