ipython notebooks: second impressions

I've kept working with ipython's notebook mode; it really is very well suited to my python-as-a-scratchpad style of figuring certain things out. So I've come up with a few more comments:
  • SVG plots work fine and look better, at least for modestly-sized output (thousands of points on a plot, megapixel images). They're also bigger on the screen, for some reason.
  • Be careful with commands that generate a lot of output; they can make your browser grind to a halt while it tries to render the output of that debugging print statement in that inner loop. During this time you also can't save your document. It might be nice if ipython noticed giant output and collapsed it.
  • It would sometimes be nice to be able to collapse blocks down to a little plus sign. For example, the first block normally needs to be the imports and maybe some ipython boilerplate (setting SVG mode, for example). It'd be nice to be able to hide it while working on the rest of the document.
  • It'd be nice to be able to break up class definitions into multiple blocks (for example to be able to have rendered math explanations for each method).
  • In PNG mode, the images are included using data URIs. This doesn't work on the very old web browser I have at work.
  • If you start pushing the memory limits of your machine it can be very hard to interact with the browser enough to kill the kernel.
For me, the most important of these is embedding the output in a blog post. It already sort of works: SVG is embedded in the output HTML, and PNG is encoded in a data: URI, so it's almost enough to strip out all the contents of the body tag and paste them into an HTML editor. Of course proper display depends on CSS, but I've added all the CSS that seemed relevant into the "custom CSS" field of my blog. The output still has some weird spacing issues; in particular, input blocks are all the same height for some reason.

Edited to add: notebooks have improved a lot since this review. I'm going to write another soon.

Blog inclusion test

This is designed to test the feasibility of inclusion of ipython notebooks into blog entries. First some math: x2,
1zdz

Now some code:
In [3]:
cfg.figure_format = 'svg' # 'png' to switch back
from IPython.zmq.pylab import backend_inline
cfg = backend_inline.InlineBackendConfig.instance()
cfg.figure_format = 'svg' # 'png' to switch back
 
import numpy as np
import matplotlib.pyplot as plt
In [4]:
8+7
print "Hello!"
8+7
Out[4]:
15
Hello!
In [5]:
plt.plot(np.linspace(0,1)**2);
plt.plot(np.linspace(0,1)**2);
In [6]:
cfg.figure_format = 'svg' # 'png' to switch back
cfg.figure_format = 'svg' # 'png' to switch back
 
In [7]:
plt.plot(np.linspace(0,1)**2);
plt.plot(np.linspace(0,1)**2);
In [ ]:

 
() [] {} 

6 comments:

Anonymous said...

Rendering is completely screwed up in the RSS feed.

Alok said...

Same here: Google Reader messes up the display completely, and I see a lot of whitespace even in the browser (Chrome 14).

(On the other hand, I really should try IPython 0.11.)

Unknown said...

I know about the whitespace - still haven't figured out how to fix it. And RSS doesn't get to use CSS, so there's not much hope that it will turn out okay. If I'd thought of it I'd have put a jump before the inclusion.

Unknown said...

Oops, both images are SVG; I meant to put a PNG in there for comparison. Sorry.

Fernando Perez said...

Anne, thanks for the second look and further feedback. Some comments:

- the svg vs png choice is tricky: they look better but for complex ones, can be huge and take forever to render. The png ones have the advantage of being fixed-size once rendered in-kernel. We should perhaps add a magic to make it easier to toggle that quickly.

- The reason they look different in size is (I suspect) a mismatch between matplotlib's DPI settings and the browser's. We've hardcoded the png dpi rendering at 72

https://github.com/ipython/ipython/blob/master/IPython/lib/pylabtools.py#L104

because Qt makes that assumption and won't let us change it. Perhaps we should change that, or at least make it user-configurable.

- Big output: good point. Yes, we'll need to refine the UI for these cases.

- Collapsing: we plan on allowing 'tabbed worksheets' (think worksheets in an excel or google docs spreadsheet). This would make it much easier to have a whole worksheet that has all initialization, data loading, etc, that you can execute once and then move away from. The current 'run all' button will need to get smarter so you can either 'run all in worksheet' or 'run all worksheets'. Furthermore, we want to expose heading levels as first-class entities (this will also let us bypass some complexities of reST heading handling). This will also eventually let us build section folding, section navigation, and commands like 'run all section' to quicly re-execute all cells in a given hierarchical level.

- Breaking up class definitions: I'm not too sure about this one. Python itself prevents that from being valid, in the sense that the whole class must be exec()'d in one shot. We could have the notion of 'linked cells', that appear split in the UI and allow text in between but always get executed in a single shot regardless of where shift-enter is pressed. But the complexity UI-wise of building this is high, so don't hold your breath on this feature.

- Sorry about the png embedding, but the other option was to serve them as files, and our entire architecture doesn't really support that, I'm afraid.

- Memory issues: yup. Granted, once you enter a bad swap storm even getting a hold of a terminal can be tricky. But I guess we could make it easier to get the PID of the kernels, so one could issue an emergency 'kill -9 PID' from a terminal before the whole thing grinds to a halt.

Please feel free to file issues for any of these you feel are most important to you! Tracker at https://github.com/ipython/ipython/issues.

so we'd need to basically

Thomas K said...

I'm impressed that it works on a 'very old web browser' at all. Testing is mostly with the latest Firefox and Chrome.