This is the last post of this year, so I try to do my best to give you something interesting to think about...
+In this case, I will show you my git workflow... and you know there are a lot of workflows out there... and probably better than mine, but I just want to share with you the place where I find myself comfortable.
+And yes... my git workflow is also powered by IPython (I am very repetitive when I love a project!). And it is a semi-automatic one, using the IPython notebooks (ipynbs) as a sort of templates, transforming them into a new conceptual entity: the ipytmpl (and yes, I love to invent names too!).
+Because my workflow have essentially two cycles, in this post, I will show you the general set up of the environment and the first Short cycle, leaving the second Extended cycle (and other details) for other post (after the new year, of course).
+I will also show you my workflow with a real PR (pull-request) to the IPython project.
+Are you ready? Here we go...
+First, we need to set up the environment to make our work:
+%pwd
+!mkdir devel_example
+%cd devel_example/
+++NOTE: You can avoid these steps opening the notebook in the proper directory, but in this case I want to have the example isolated to not overwrite my current development environment.
+
project_name = "ipython"
+project_remote = "git://github.com/ipython/ipython.git"
+project_remote_name = "origin"
+my_fork_remote = "git@github.com:damianavila/ipython.git"
+my_fork_remote_name = "damianavila"
+# Get a read-only copy of the project
+!git clone $project_remote
+
+# cd into the local project folder
+%cd $project_name
+
+# Link the local repo with my Github fork
+!git remote add $my_fork_remote_name $my_fork_remote
+
+# Check remotes
+!git remote -v
+++NOTE: A lot of git workflows use
+originto point to our fork andupstreamto point to the project repo. But I do not like that configuration. It seems more natural to me to clone the project repo (theoriginrepo) and add a connection to my fork calleddamianavila... and the next steps take into consideration this last approach.
This short cycle just create a new branch to work on, make the needed changes in the source code and upload the local changes to our Github fork to finally submit a pull-request:
+master and development branch names:master_branch = "master"
+feature_branch = "doc_post_serve"
+master:# Make sure we are in master branch
+!git checkout $master_branch
+
+# Pull the changes from origin/master
+!git pull $project_remote_name
+
+# Start a new branch to work on
+!git checkout -b $feature_branch
+
+# Check where we are
+!git status
+++NOTE: In this example, I will update the IPython docs about some details using the IPython slides and the
+post-servepost-processor (IPython.nbconvert).
# list the files structure to find the needed files
+%ls
+%load docs/source/interactive/nbconvert.rst
+# After executing %load, a new cell containing the source code will be added.
+# Be sure to add the next line (with the proper path) to overwrite the file
+# with you changes.
+#
+# %%writefile docs/source/interactive/nbconvert.rst
+%%writefile docs/source/interactive/nbconvert.rst
+.. _nbconvert:
+
+Converting notebooks to other formats
+=====================================
+
+Newly added in the 1.0 release of IPython is the ``nbconvert`` tool, which
+allows you to convert an ``.ipynb`` notebook document file into various static
+formats.
+
+Currently, ``nbconvert`` is provided as a command line tool, run as a script
+using IPython. A direct export capability from within the
+IPython Notebook web app is planned.
+
+The command-line syntax to run the ``nbconvert`` script is::
+
+ $ ipython nbconvert --to FORMAT notebook.ipynb
+
+This will convert the IPython document file ``notebook.ipynb`` into the output
+format given by the ``FORMAT`` string.
+
+The default output format is html, for which the ``--to`` argument may be
+omitted::
+
+ $ ipython nbconvert notebook.ipynb
+
+IPython provides a few templates for some output formats, and these can be
+specified via an additional ``--template`` argument.
+
+The currently supported export formats are:
+
+* ``--to html``
+
+ - ``--template full`` (default)
+
+ A full static HTML render of the notebook.
+ This looks very similar to the interactive view.
+
+ - ``--template basic``
+
+ Simplified HTML, useful for embedding in webpages, blogs, etc.
+ This excludes HTML headers.
+
+* ``--to latex``
+
+ Latex export. This generates ``NOTEBOOK_NAME.tex`` file,
+ ready for export. You can automatically run latex on it to generate a PDF
+ by adding ``--post PDF``.
+
+ - ``--template article`` (default)
+
+ Latex article, derived from Sphinx's howto template.
+
+ - ``--template book``
+
+ Latex book, derived from Sphinx's manual template.
+
+ - ``--template basic``
+
+ Very basic latex output - mainly meant as a starting point for custom templates.
+
+* ``--to slides``
+
+ This generates a Reveal.js HTML slideshow.
+ It must be served by an HTTP server. The easiest way to do this is adding
+ ``--post serve`` on the command-line. The ``--post serve`` post-processor
+ proxies Reveal.js requests to a CDN if no local Reveal.js library is present.
+ For low connectivity environments, just place the Reveal.js library in the
+ same directory where your_talk.slides.html is located or point to another
+ directory using the ``--reveal-prefix`` alias.
+
+* ``--to markdown``
+
+ Simple markdown output. Markdown cells are unaffected,
+ and code cells are placed in triple-backtick (```````) blocks.
+
+* ``--to rst``
+
+ Basic reStructuredText output. Useful as a starting point for embedding notebooks
+ in Sphinx docs.
+
+* ``--to python``
+
+ Convert a notebook to an executable Python script.
+ This is the simplest way to get a Python script out of a notebook.
+ If there were any magics in the notebook, this may only be executable from
+ an IPython session.
+
+.. note::
+
+ nbconvert uses pandoc_ to convert between various markup languages,
+ so pandoc is a dependency of most nbconvert transforms,
+ excluding Markdown and Python.
+
+.. _pandoc: http://johnmacfarlane.net/pandoc/
+
+The output file created by ``nbconvert`` will have the same base name as
+the notebook and will be placed in the current working directory. Any
+supporting files (graphics, etc) will be placed in a new directory with the
+same base name as the notebook, suffixed with ``_files``::
+
+ $ ipython nbconvert notebook.ipynb
+ $ ls
+ notebook.ipynb notebook.html notebook_files/
+
+For simple single-file output, such as html, markdown, etc.,
+the output may be sent to standard output with::
+
+ $ ipython nbconvert --to markdown notebook.ipynb --stdout
+
+Multiple notebooks can be specified from the command line::
+
+ $ ipython nbconvert notebook*.ipynb
+ $ ipython nbconvert notebook1.ipynb notebook2.ipynb
+
+or via a list in a configuration file, say ``mycfg.py``, containing the text::
+
+ c = get_config()
+ c.NbConvertApp.notebooks = ["notebook1.ipynb", "notebook2.ipynb"]
+
+and using the command::
+
+ $ ipython nbconvert --config mycfg.py
+
+
+.. _notebook_format:
+
+LaTeX citations
+---------------
+
+``nbconvert`` now has support for LaTeX citations. With this capability you
+can:
+
+* Manage citations using BibTeX.
+* Cite those citations in Markdown cells using HTML data attributes.
+* Have ``nbconvert`` generate proper LaTeX citations and run BibTeX.
+
+For an example of how this works, please see the citations example in
+the nbconvert-examples_ repository.
+
+.. _nbconvert-examples: https://github.com/ipython/nbconvert-examples
+
+Notebook JSON file format
+-------------------------
+
+Notebook documents are JSON files with an ``.ipynb`` extension, formatted
+as legibly as possible with minimal extra indentation and cell content broken
+across lines to make them reasonably friendly to use in version-control
+workflows. You should be very careful if you ever manually edit this JSON
+data, as it is extremely easy to corrupt its internal structure and make the
+file impossible to load. In general, you should consider the notebook as a
+file meant only to be edited by the IPython Notebook app itself, not for
+hand-editing.
+
+.. note::
+
+ Binary data such as figures are also saved directly in the JSON file.
+ This provides convenient single-file portability, but means that the
+ files can be large; a ``diff`` of binary data is also not very
+ meaningful. Since the binary blobs are encoded in a single line, they
+ affect only one line of the ``diff`` output, but they are typically very
+ long lines. You can use the ``Cell | All Output | Clear`` menu option to
+ remove all output from a notebook prior to committing it to version
+ control, if this is a concern.
+
+The notebook server can also generate a pure Python version of your notebook,
+using the ``File | Download as`` menu option. The resulting ``.py`` file will
+contain all the code cells from your notebook verbatim, and all Markdown cells
+prepended with a comment marker. The separation between code and Markdown
+cells is indicated with special comments and there is a header indicating the
+format version. All output is removed when exporting to Python.
+
+As an example, consider a simple notebook called ``simple.ipynb`` which
+contains one Markdown cell, with the content ``The simplest notebook.``, one
+code input cell with the content ``print "Hello, IPython!"``, and the
+corresponding output.
+
+The contents of the notebook document ``simple.ipynb`` is the following JSON
+container::
+
+ {
+ "metadata": {
+ "name": "simple"
+ },
+ "nbformat": 3,
+ "nbformat_minor": 0,
+ "worksheets": [
+ {
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": "The simplest notebook."
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": "print \"Hello, IPython\"",
+ "language": "python",
+ "metadata": {},
+ "outputs": [
+ {
+ "output_type": "stream",
+ "stream": "stdout",
+ "text": "Hello, IPython\n"
+ }
+ ],
+ "prompt_number": 1
+ }
+ ],
+ "metadata": {}
+ }
+ ]
+ }
+
+
+The corresponding Python script is::
+
+ # -*- coding: utf-8 -*-
+ # <nbformat>3.0</nbformat>
+
+ # <markdowncell>
+
+ # The simplest notebook.
+
+ # <codecell>
+
+ print "Hello, IPython"
+
+Note that indeed the output of the code cell, which is present in the JSON
+container, has been removed in the ``.py`` script.
+# Check status
+!git status
+# See the diff
+!git diff
+# Add the modified files to the stage
+!git add .
+# And do your commit
+!git commit -am "Added --post-serve explanation into the nbconvert docs."
+# Push updates from your local branch to your github branch
+!git push $my_fork_remote_name $feature_branch
+++NOTE: The merging of your Github development branch into the master is done via pull-request on the Github website. For reference, you can see the proposed PR here: https://github.com/ipython/ipython/pull/4751
+
As you can see, this workflow is very simple... and with the aid of this ipytmpl is easier than before (before = making the same but in your traditional console).
+You set up the environment, fill the variables to use a posteriori, and you have only to be concern about the changes you want to introduce (or remove) from the source code. All the other steps, all those git calls are predetermined and will be called whereas you advance in the workflow...
+After making the PR at the Github website, you will receive some feedback and if you have to modified something, just start the short cycle again... Sometimes you will need more... I mean, because you are working in a communitary project, if somebody changes the same file as you, there will be some conflicts at the merge step, so it will be necessary to rebase the "thing". But this is the central idea of the second Extended cycle which I will describe you in a second part of this post.
+As always, I am waiting for your comments and critics!
+OK, too long... I hope you did not get bored!
+Have a nice New Year! And I see you in 2014 ;-)
+Cheers.
+Damián
+This is the last post of this year, so I try to do my best to give you something interesting to think about...
+In this case, I will show you my git workflow... and you know there are a lot of workflows out there... and probably better than mine, but I just want to share with you the place where I find myself comfortable.
+And yes... my git workflow is also powered by IPython (I am very repetitive when I love a project!). And it is a semi-automatic one, using the IPython notebooks (ipynbs) as a sort of templates, transforming them into a new conceptual entity: the ipytmpl (and yes, I love to invent names too!).
+Because my workflow have essentially two cycles, in this post, I will show you the general set up of the environment and the first Short cycle, leaving the second Extended cycle (and other details) for other post (after the new year, of course).
+I will also show you my workflow with a real PR (pull-request) to the IPython project.
+Are you ready? Here we go...
+First, we need to set up the environment to make our work:
+%pwd
+!mkdir devel_example
+%cd devel_example/
+++NOTE: You can avoid these steps opening the notebook in the proper directory, but in this case I want to have the example isolated to not overwrite my current development environment.
+
project_name = "ipython"
+project_remote = "git://github.com/ipython/ipython.git"
+project_remote_name = "origin"
+my_fork_remote = "git@github.com:damianavila/ipython.git"
+my_fork_remote_name = "damianavila"
+# Get a read-only copy of the project
+!git clone $project_remote
+
+# cd into the local project folder
+%cd $project_name
+
+# Link the local repo with my Github fork
+!git remote add $my_fork_remote_name $my_fork_remote
+
+# Check remotes
+!git remote -v
+++NOTE: A lot of git workflows use
+originto point to our fork andupstreamto point to the project repo. But I do not like that configuration. It seems more natural to me to clone the project repo (theoriginrepo) and add a connection to my fork calleddamianavila... and the next steps take into consideration this last approach.
This short cycle just create a new branch to work on, make the needed changes in the source code and upload the local changes to our Github fork to finally submit a pull-request:
+master and development branch names:master_branch = "master"
+feature_branch = "doc_post_serve"
+master:# Make sure we are in master branch
+!git checkout $master_branch
+
+# Pull the changes from origin/master
+!git pull $project_remote_name
+
+# Start a new branch to work on
+!git checkout -b $feature_branch
+
+# Check where we are
+!git status
+++NOTE: In this example, I will update the IPython docs about some details using the IPython slides and the
+post-servepost-processor (IPython.nbconvert).
# list the files structure to find the needed files
+%ls
+%load docs/source/interactive/nbconvert.rst
+# After executing %load, a new cell containing the source code will be added.
+# Be sure to add the next line (with the proper path) to overwrite the file
+# with you changes.
+#
+# %%writefile docs/source/interactive/nbconvert.rst
+%%writefile docs/source/interactive/nbconvert.rst
+.. _nbconvert:
+
+Converting notebooks to other formats
+=====================================
+
+Newly added in the 1.0 release of IPython is the ``nbconvert`` tool, which
+allows you to convert an ``.ipynb`` notebook document file into various static
+formats.
+
+Currently, ``nbconvert`` is provided as a command line tool, run as a script
+using IPython. A direct export capability from within the
+IPython Notebook web app is planned.
+
+The command-line syntax to run the ``nbconvert`` script is::
+
+ $ ipython nbconvert --to FORMAT notebook.ipynb
+
+This will convert the IPython document file ``notebook.ipynb`` into the output
+format given by the ``FORMAT`` string.
+
+The default output format is html, for which the ``--to`` argument may be
+omitted::
+
+ $ ipython nbconvert notebook.ipynb
+
+IPython provides a few templates for some output formats, and these can be
+specified via an additional ``--template`` argument.
+
+The currently supported export formats are:
+
+* ``--to html``
+
+ - ``--template full`` (default)
+
+ A full static HTML render of the notebook.
+ This looks very similar to the interactive view.
+
+ - ``--template basic``
+
+ Simplified HTML, useful for embedding in webpages, blogs, etc.
+ This excludes HTML headers.
+
+* ``--to latex``
+
+ Latex export. This generates ``NOTEBOOK_NAME.tex`` file,
+ ready for export. You can automatically run latex on it to generate a PDF
+ by adding ``--post PDF``.
+
+ - ``--template article`` (default)
+
+ Latex article, derived from Sphinx's howto template.
+
+ - ``--template book``
+
+ Latex book, derived from Sphinx's manual template.
+
+ - ``--template basic``
+
+ Very basic latex output - mainly meant as a starting point for custom templates.
+
+* ``--to slides``
+
+ This generates a Reveal.js HTML slideshow.
+ It must be served by an HTTP server. The easiest way to do this is adding
+ ``--post serve`` on the command-line. The ``--post serve`` post-processor
+ proxies Reveal.js requests to a CDN if no local Reveal.js library is present.
+ For low connectivity environments, just place the Reveal.js library in the
+ same directory where your_talk.slides.html is located or point to another
+ directory using the ``--reveal-prefix`` alias.
+
+* ``--to markdown``
+
+ Simple markdown output. Markdown cells are unaffected,
+ and code cells are placed in triple-backtick (```````) blocks.
+
+* ``--to rst``
+
+ Basic reStructuredText output. Useful as a starting point for embedding notebooks
+ in Sphinx docs.
+
+* ``--to python``
+
+ Convert a notebook to an executable Python script.
+ This is the simplest way to get a Python script out of a notebook.
+ If there were any magics in the notebook, this may only be executable from
+ an IPython session.
+
+.. note::
+
+ nbconvert uses pandoc_ to convert between various markup languages,
+ so pandoc is a dependency of most nbconvert transforms,
+ excluding Markdown and Python.
+
+.. _pandoc: http://johnmacfarlane.net/pandoc/
+
+The output file created by ``nbconvert`` will have the same base name as
+the notebook and will be placed in the current working directory. Any
+supporting files (graphics, etc) will be placed in a new directory with the
+same base name as the notebook, suffixed with ``_files``::
+
+ $ ipython nbconvert notebook.ipynb
+ $ ls
+ notebook.ipynb notebook.html notebook_files/
+
+For simple single-file output, such as html, markdown, etc.,
+the output may be sent to standard output with::
+
+ $ ipython nbconvert --to markdown notebook.ipynb --stdout
+
+Multiple notebooks can be specified from the command line::
+
+ $ ipython nbconvert notebook*.ipynb
+ $ ipython nbconvert notebook1.ipynb notebook2.ipynb
+
+or via a list in a configuration file, say ``mycfg.py``, containing the text::
+
+ c = get_config()
+ c.NbConvertApp.notebooks = ["notebook1.ipynb", "notebook2.ipynb"]
+
+and using the command::
+
+ $ ipython nbconvert --config mycfg.py
+
+
+.. _notebook_format:
+
+LaTeX citations
+---------------
+
+``nbconvert`` now has support for LaTeX citations. With this capability you
+can:
+
+* Manage citations using BibTeX.
+* Cite those citations in Markdown cells using HTML data attributes.
+* Have ``nbconvert`` generate proper LaTeX citations and run BibTeX.
+
+For an example of how this works, please see the citations example in
+the nbconvert-examples_ repository.
+
+.. _nbconvert-examples: https://github.com/ipython/nbconvert-examples
+
+Notebook JSON file format
+-------------------------
+
+Notebook documents are JSON files with an ``.ipynb`` extension, formatted
+as legibly as possible with minimal extra indentation and cell content broken
+across lines to make them reasonably friendly to use in version-control
+workflows. You should be very careful if you ever manually edit this JSON
+data, as it is extremely easy to corrupt its internal structure and make the
+file impossible to load. In general, you should consider the notebook as a
+file meant only to be edited by the IPython Notebook app itself, not for
+hand-editing.
+
+.. note::
+
+ Binary data such as figures are also saved directly in the JSON file.
+ This provides convenient single-file portability, but means that the
+ files can be large; a ``diff`` of binary data is also not very
+ meaningful. Since the binary blobs are encoded in a single line, they
+ affect only one line of the ``diff`` output, but they are typically very
+ long lines. You can use the ``Cell | All Output | Clear`` menu option to
+ remove all output from a notebook prior to committing it to version
+ control, if this is a concern.
+
+The notebook server can also generate a pure Python version of your notebook,
+using the ``File | Download as`` menu option. The resulting ``.py`` file will
+contain all the code cells from your notebook verbatim, and all Markdown cells
+prepended with a comment marker. The separation between code and Markdown
+cells is indicated with special comments and there is a header indicating the
+format version. All output is removed when exporting to Python.
+
+As an example, consider a simple notebook called ``simple.ipynb`` which
+contains one Markdown cell, with the content ``The simplest notebook.``, one
+code input cell with the content ``print "Hello, IPython!"``, and the
+corresponding output.
+
+The contents of the notebook document ``simple.ipynb`` is the following JSON
+container::
+
+ {
+ "metadata": {
+ "name": "simple"
+ },
+ "nbformat": 3,
+ "nbformat_minor": 0,
+ "worksheets": [
+ {
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": "The simplest notebook."
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": "print \"Hello, IPython\"",
+ "language": "python",
+ "metadata": {},
+ "outputs": [
+ {
+ "output_type": "stream",
+ "stream": "stdout",
+ "text": "Hello, IPython\n"
+ }
+ ],
+ "prompt_number": 1
+ }
+ ],
+ "metadata": {}
+ }
+ ]
+ }
+
+
+The corresponding Python script is::
+
+ # -*- coding: utf-8 -*-
+ # <nbformat>3.0</nbformat>
+
+ # <markdowncell>
+
+ # The simplest notebook.
+
+ # <codecell>
+
+ print "Hello, IPython"
+
+Note that indeed the output of the code cell, which is present in the JSON
+container, has been removed in the ``.py`` script.
+# Check status
+!git status
+# See the diff
+!git diff
+# Add the modified files to the stage
+!git add .
+# And do your commit
+!git commit -am "Added --post-serve explanation into the nbconvert docs."
+# Push updates from your local branch to your github branch
+!git push $my_fork_remote_name $feature_branch
+++NOTE: The merging of your Github development branch into the master is done via pull-request on the Github website. For reference, you can see the proposed PR here: https://github.com/ipython/ipython/pull/4751
+
As you can see, this workflow is very simple... and with the aid of this ipytmpl is easier than before (before = making the same but in your traditional console).
+You set up the environment, fill the variables to use a posteriori, and you have only to be concern about the changes you want to introduce (or remove) from the source code. All the other steps, all those git calls are predetermined and will be called whereas you advance in the workflow...
+After making the PR at the Github website, you will receive some feedback and if you have to modified something, just start the short cycle again... Sometimes you will need more... I mean, because you are working in a communitary project, if somebody changes the same file as you, there will be some conflicts at the merge step, so it will be necessary to rebase the "thing". But this is the central idea of the second Extended cycle which I will describe you in a second part of this post.
+As always, I am waiting for your comments and critics!
+OK, too long... I hope you did not get bored!
+Have a nice New Year! And I see you in 2014 ;-)
+Cheers.
+Damián
+
+
+
+
+
+
+
+
+
+