The idea was to create AI intentions – code suggestions based on what you’ve just written. They cover a wide range of situations from generating code to warnings and optimization suggestions. The main goal was to develop NLP model and rank the list of predicted intentions based on notebook content.

I managed this project and designed the architecture to build a complete working prototype.


Firstly, 6 mln jupyter notebooks had been parsed. We ended up with 3.4 mln notebooks after filtering out irrelevant ones (empty, Python 2).

Dataset sample

After EDA, we got some insights:

  • 2.6% of notebooks don't use Python – there're also R, Julia and Scala kernels along with an insignificant number of more exotic ones (C++, Haskell, OCaml, F#)
  • In the last 2 years, the number of Python 3 notebooks grew by 660%
  • > 300.000 notebooks with sensitive data

Model implementation

AI Intentions module implemented as two standalone services for processing data from/to user and making predictions (CPU- and GPU-intensive, scaled independently)

The predicting model: each user-side event is mapped to the learned embedding vector, the result is passed to a deep LSTM network; finally, dense bottleneck maps LSTM output features to the probabilities of the next intention

Tech stack

  • TensorFlow 2.0 for training models
  • TensorFlow Serving
  • Flask (via uwsgi-nginx-flask) for pre/post processing data


  • Input – events generated by user are mapped to learned embeddings, which are passed to the self-attention layer
  • Output was passed to the deep LSTM (each layer followed by LayerNorm)
  • LSTM outputs are passed to the dense bottleneck layer, regularized with low-rate dropout


AI Intentions models obtain up to:

  • 60%@top-1 accuracy
  • 92%@top-5 accuracy


  • Adam Hood, Software Developer
  • Vladimir Sotnikov, Data Scientist

My talk at PyConDE & PyData Berlin:

Back to Portfolio