S3-backed notebook manager for IPython

Setup

1. Install:

   from pypi:

   pip install s3nb

   from source with pip:

   pip install git+https://github.com/monetate/s3nb

   or from source the old fashioned way:

   git clone git@github.com:monetate/s3nb.git
   cd s3nb
   python ./setup.py install

2. Configure

   # set this - notebooks will be stored relative to this uri
   S3_NOTEBOOK_URI=s3://path/to/notebooks/
   # and this
   IPYTHON_MAJOR_VERSION=2
   # optionally set this - checkpoints will be stored locally, relative to this path (for IPython 3)
   CHECKPOINT_ROOT_DIR=~/.checkpoints
   # optionally set this
   PROFILE=s3nbserver
   # shouldn't need to edit beyond this point## IPython 2.x
   IPYNB_MANAGER=S3NotebookManager
   IPYNB_MANAGER_CFG=notebook_manager_class
   ## IPython 3.xif [ $IPYTHON_MAJOR_VERSION== 3 ];then
   IPYNB_MANAGER=S3ContentsManager
   IPYNB_MANAGER_CFG=contents_manager_class
   fi
   IPYTHONDIR=${IPYTHONDIR:-$HOME/.ipython}
   PROFILE_DIR=${IPYTHONDIR}/profile_${PROFILE}if [ !-d$PROFILE_DIR ];then
   ipython profile create $PROFILE
   IPYNB_CONFIG=${PROFILE_DIR}/ipython_notebook_config.py
   mv $IPYNB_CONFIG$IPYNB_CONFIG.orig
   cat >$IPYNB_CONFIG<<EOFc = get_config()c.NotebookApp.${IPYNB_MANAGER_CFG} = 's3nb.${IPYNB_MANAGER}'c.${IPYNB_MANAGER}.s3_base_uri = '$S3_NOTEBOOK_URI'EOFfiif [ $IPYTHON_MAJOR_VERSION== 3 ];thenecho"c.S3ContentsManager.checkpoints_kwargs = {'root_dir': '${CHECKPOINT_ROOT_DIR}'}">>${IPYNB_CONFIG}fi

3. If you haven't already, configure AWS variables for boto. Follow these instructions.

4. Run

   ipython notebook --profile=s3nbserver

Development

1. Provision a virtual machine with vagrant up
2. Create an IPython profile with make configure -e S3_BASE_URI=YOUR_BUCKET
3. Share you AWS credentials with the virtual machine with make configure -e AWS_USER=YOUR_USER
4. Run the notebook server with make run