Newsletter issue #1 - Our bright (and not in the least bit scary) AI Agents future
Six months ago AI Agents were “in the future”, in the past month agents and assistants powered by AI have become a reality.
Welcome to this, the first issue of the Time Under Tension email newsletter! We plan to publish roughly every one to two weeks, with an opinion piece plus a handful of super-interesting Gen AI nuggets. This was originally published on Substack, where you can subscribe for any future emails.
AI Agents, back to the future
For the last six months, the final slide of the Time Under Tension Inform presentation has been a glimpse into the future of generative AI agents;
In this example, Joshua Browder “outsourced his personal finance life” to an AI agent. Using AutoGPT (an open-source project) Joshua gave the agent access to his bank account, financial statements, email and more. The agent was given a specific goal; save Joshua money, and when released it saved him $217.86 in the first 24 hours alone, by cancelling subscriptions and requesting refunds on his behalf.
This was an attention-getting demonstration of the potential of agents to be given instructions by their human overlords, and follow those instructions outside of the chat sandbox of ChatGPT, Bing or Bard.
Six months ago this was “the future”, in the past month agents and assistants (in this post we’ll use these terms interchangeably) powered by AI have become a reality.
Why now?
There have been many breakthroughs in agents since Joshua’s post in April 2023, but two stand out in my mind. Firstly, an impactful article by Bill Gates in early November described how he thinks;
“Agents are not only going to change how everyone interacts with computers. They’re also going to upend the software industry, bringing about the biggest revolution in computing since we went from typing commands to tapping on icons.”
- Bill Gates
The second, is the release in November by OpenAI of their Assistants API, which makes the development of agents much easier than it was before. We have been experimenting with this, and an example is below.
In short, the use of assistants is now easier to imagine thanks to Bill Gates, and the creation of them is now easier thanks to OpenAI.
How would I use an agent?
Businesses are able to use agents for both internal and external purposes.
An example of an external use could be an eCommerce retailer such as Bunnings providing a personalised shopping assistant to customers. The assistant could automatically price check against other retailers, and offer to beat their prices by 10%.
An internal use might be, as we demonstrate below, a tool that writes email newsletters for you. Our demo uses two agents to research and collate material for a newsletter, draft an email and (once approved by a human) integrate with an external mailer platform to publish it.
Agents in action: our demo
Here at Time Under Tension we build generative AI software for our clients, and as part of our R&D wanted to get our hands dirty with the newly released Assistants API from OpenAI.
The scenario we built is a common one; writing and sending an email marketing newsletter.
The assistant we built has a conversational (chat) UX, that works like this;
The (human!) operator can instruct Assistant #1 (‘Content Producer’) to get started, and how much content to include in the newsletter
Assistant #1 will scrape the Time Under Tension blog for articles
The Assistant will collate and summarise the articles
The human operator can ask for changes in the content, which the Assistant will make
Once everything looks ok, the human approves the content as a draft
Next, a new AI Assistant #2 (‘Marketing Manager’) is introduced, which is integrated with MailChimp and will request a draft email to be sent for review
The human receives an email, and can then ask Assistant #2 to make changes, or approve it
Once approved, Assistant #2 will publish the newsletter to MailChimp and it will be sent to the mailing list
Watch it in action (video has been sped up for brevity)
Ordinarily, to have multiple ‘personas’ and integration points would require a lot of custom development, however with the OpenAI Assistants API it’s now much easier.
Agents are great at determining the necessary sequence of steps to achieve a given task.
They can be given “toolboxes” specific to their purpose and are knowledgeable about when and where to use these tools (for example, the Content Producer and Marketing Manager are given different instructions, but a common goal).
An assistant for everyone
Building assistants is easier now that it has ever been, but it’s very new technology and not without its difficulties.
Our advice to our clients is to consider what use-cases agents might unlock (we can help with this ideation), and think about getting started with a rapid proof-of-concept (like our demo above which took a week to build).
There are enormous productivity gains to be had, and new experiences to offer to customers. Agents will undoubtedly play a larger role in the future, and the technology is available here and now.
Agents: technical details
Our demo was built with the OpenAI Assistants API, with a React Front-end and a Node.js backend.
Learnings:
Agents/Assistants are great at determining the necessary sequence of steps to achieve a given task.
Managing threads is key. A thread is OpenAI’s version of a chain of messages. Threads can contain multiple agents (only one at a time though). You can start a thread with one agent and then continue chatting in this same thread with a different agent. This is useful because the new agent now has the entire conversation history.
You need to be smart about how you create your agent tools. You don’t want to pack too many steps into a single tool as if that tool throws an error, the agent will try and re-run the entire tool. It’s better to separate things out: one tool for one job!
Pros:
Absolutely amazing at chain-of-logic. Knows when to use a tool, when to use several tools, when it might need to try something again, knows how to interpret errors.
Cons:
Multi-agent interactions is not quite trivial at the moment. There’s lots of challenges that arise by letting agents communicate independently. Who starts the conversation? Who determines who should speak? Who steps in if the conversation gets derailed?
A handful of Gen AI news
Here are five of the most interesting things we have seen and read in the last week;
Australian image generation start-up Leonardo.AI just raised $47m
A great post from Ben’s Bites on how to build an AI powered company from the ground up. Best read alongside this HBR article ‘Strategy, Not Technology, Is the Key to Winning with GenAI’
Magical emoji >> photos created using magnific.ai - see the post on X for more emoji sleight of hand… 🫣 🚗 🤡 🍔 🍓 📷
Interesting research from O’Rielly on Enterprise take-up of generative AI. I would take the stats with a grain of salt (survey bias) but the section ‘What’s Holding AI Back?’ rings true
This unofficial Adidas commercial was entirely generated with AI by @Theblasian35, using a combination of Midjourney, Runway, and Kaiber. Generative AI for video is moving so fast
That’s all for issue #1, we hope you liked it!