Data Science Automation

Problem

To what extent can we automate data science tasks, such as market analysis? Solving data science problems is a complex and open ended problem, we cannot give a plan for how to solve it. Furthermore, data scientists need to co-operate and code in a shared environment, which was not possible for AI agents at the time of this project. However, if this problem could be solved by ai agents then it could be attempted in parallel, allowing agents to explore the many possibilities which a human scientist has no time for.

Solution

This required a framework which grants true decision making agency to AI models. Microsoft Research had just launched Autogen and Agentico used it to arrange AI agents into a data science team. The team would accept any objective, exploring whatever data is provided to them and apply appropriate algorithms without human guidance.

Our unique innovation was to allow the agents to collaboratively code in the data scientist’s favourite tool, a Jupyter notebook. Wherein they could plan together , critique each other’s plans, explore and clean data, execute code and rectify errors.

The solution was so successful that Microsoft Research built it into their AutoGen project. However, it does have limits. The agent team must work within their context window.

Recipe

• AI Agent Framework:

  • Microsoft Autogen

    
    

• There were four roles in the agent team:

  • A data scientist / python developer

  • A critic to review the coder's work

  • A conversation manager, intelligently choosing which of the participants should speak or act next

  • The Jupyter Notebook was itself an ‘agent’ albeit just a set of rules, not an LLM.

We then developed a customisation to the base Autogen package which allows agents to:

• add markdown cells to the notebook

• to discuss plans as a team

• add code cells, which are then automatically executed and results passed to the agents

The conversation manager generates a 'definition of done' and decides when to end the project.