Skafos + Jupyter = Better Together
Jupyter hosted on Skafos gives you the full power of the Skafos platform powering JupyterLab.
Project Jupyter is a non-profit, open-source project to support interactive data science and scientific computing. JupyterLab is an interactive development environment that has full support for Jupyter Notebooks, is highly customizable, and provides integration between notebooks and code.
Skafos provisions a JupyterLab instance for every Skafos Project you create. Using JupyterLab is not required, but it can be an incredibly powerful tool for writing machine learning models.
When you create a new Project, a fresh lab instance is automatically provisioned for you to begin your development. To access this lab instance, navigate to the Dashboard and click on the Launch JupyterLab button next to the project of interest. Once inside your lab workspace, you can:
- write and test new code for jobs within your project
- easily install your favorite packages or dependencies in the terminal
- interact with the Skafos SDK and Data Engine
- track changes using git
This allows you to leverage the power of Skafos while working in a familiar development environment synced up to your personal code repository.
Jupyter Lab has many features that many users are familiar with, and are well documented in the JupyterLab Documentation. These include:
Because each of the Jupyter Lab instances provisioned by Skafos is built on our infrastructure, you have full access to the Skafos SDK, including Data & ML Model Handling through the Skafos Data Engine. This is an excellent way to prototype data access and handling prior to delivering into a production pipeline.
When you're ready to deploy your project from a notebook session, you have two options:
1) Use the terminal to push your code changes to the master branch of your associated github repostitory. (You will need to add the Skafos App to your github repository first).
2) Use the Skafos Deploy option directly from the notebook, as described below.
To deploy directly from the notebook you are using, click the Skafos link. Two options will be present:
- Open Project Config
Begin by opening the project config.
Make sure that the entry point and resources reflect what is needed for this job. (Note that if you need an additional job or dependency, this can only be achieved if you use the Command Line Interface to add jobs.
Once this file has been edited and saved, if needed, go back to the Skafos menu and select Deploy. You will receive a pop-up
Click Deploy, and you will see another pop up, and a link for where you can monitor your job.