Update June 10, 2024
It is time for another Breadboard update! The Visual Editor keeps maturing and becoming the 🦋 beautiful butterfly that it's meant to be. There's two new things called "Board Server" and "Connection Server", and of course, the 🕵️ Agent Kit!
🎨 Visual Editor
It's been a very educational week for Visual Editor. We've taught it a bunch of things. First, we gave the left-side panel an overhaul. Instead of just a long laundry list of boards to load, it is now neatly organized by the provider.
A "Board Provider" is a general term for any source of boards. At the start, all Visual Editor users get two providers: the "Board Store" and the "Example Boards". The first one is a local-only store that board makers can use to quickly throw together boards. The examples one is read-only and contains interesting examples of already made boards -- to help with the blank canvas problem.
You can add a new board provider by clicking on the icon right next to the provider drop-down menu. There are currently two types of additional providers supported: the "File System" one (you've read about before in the March update), and -- new for this update -- the "Board Server".
Remember that one. We'll talk about Board Servers in a little bit. Board Servers are super-cool ✨.
Next, we've taught Breadboard how to save and load runs. This is something we've been wanting to do for a long while. When we run a board in Breadboard Visual Editor, we see the Activity log: this really cool collection of all the telemetry that happened along the way. It's a wealth of information: turtle upon turtle of graphs and nodes. This information (also known as "the run") reveals everything that happened while invoking the board and can be very useful for diagnostics and debugging.
You can now store and replay this information. To store the run, click the "Download" link on the top right corner of the Activity log (we should probably rename it to "Run Log", right?). The Visual Editor will package up the run and download it for you in JSON format.
To replay the run, simply drag the previously saved file into the Visual Editor. You will see the run quickly play out in the Activity log, including all of the multimodal content that was supplied and created during the saved run.
Finally, we've applied a lot of effort to teach the Visual Editor manners: from responding correctly to the keyboard shortcuts on Linux to gracefully handling edge deletions, to all kinds of minor, yet-oh-so-important polish that transforms the awkward teenage kid into a thriving young adult.
🏄 Board Server
A new addition to the Breadboard bunch is the Board Server. As the name implies, Board Servers serve boards (🤯 whoa). We've been thinking for a while on how to introduce the concept of easy storing, reusing, and remixing of the boards, and here's what we came up with.
Instead of having the Visual Editor be the single place that manages the board storage, we've gone with a bit more federated approach. Any group of peeps, be it a team or just a creative collective, can stand up a board server. A board server then talks with the Visual Editor using a fairly straightforward CRUD protocol, which allows Visual Editor users to add it as a board provider.
As a simple authentication mechanism, the Board Server hosts issue API keys to the server members.
Once I add the board server, all published boards from all server members will show up in left navigation. I can now use them as my tools, my inspiration, or as starting points in my prototyping adventures.
I can be a member of more than one server. A server can be private or public. We believe that this approach gives more agency to the users of Visual Breadboard, giving the ability to decide how and where they will share their creations.
Because the board servers store boards, they are also as a great place to run them. The current Board Server implementation is still very nascent, but it already exposes two extra views of a board:
The App View: If I change the extension of the board URL from .json to .app, I will see an automatically generated preview of the board as an app. This could be super-useful for quickly sharing the board I built with others.
The API View: If I change the extension to .api, I will get an API endpoint, which allows me to call this board as an API.
We're actively working on teaching the Board Server a whole bunch of new tricks, and are very excited about its future.
🔌 Connection Server
The Connection Server is another newcomer. Similar in spirit to Board Servers, Connection Servers take a federated approach to authentication and authorization. Very often, the API we want to call from Breadboard require OAuth or similar auth protocols -- sometimes all within the same board. The Connection Server offers this capability, providing a simple, configurable service that enables signing into multiple services at the same time.
This way, if I want to use Github Gists as my agent's memory while relying on Google Drive to convert my PDFs into text, I can do that without having to joggle the multiple sign-ins.
🕵️ Agent Kit
One last big batch of work to cover is around the Agent Kit. The Agent Kit is a collection of Breadboard nodes that is specifically tailored to rapidly prototype and ship semi-autonomous, asynchronous agents. Think of each node as a body part of an agent. Each is limited in what it can do, but when put together, they can produce pretty powerful experiences.
There is an emerging guide for using the Agent Kit here. Very briefly, there are three important nodes: Specialist, Looper, and Human.
The Specialist node is the most versatile of the bunch, and can do many things, as long as it's one task. By customizing the Specialist and imbuing it with a persona (aka "system instruction") and tools (aka "function calling"), we can focus it on a particular task at hand. We can ask the Specialist to write a travel guide or search the Internet or generate a prompt and a picture from that prompt.
The Looper node allows creating iterative flows that involve planning and execution of more complex tasks. For example, we can ask the looper to create a plan for writing a book chapter-by-chapter or tell it to keep iterating on a project pitch until the user is satisfied with the edits. The looper is commonly used to organize work, like in the example below, where it's creating a series of tasks for Specialists.
The Human is the node that puts us humans in the loop. Put this node whenever we need to hear feedback from the user, to provide comments or pick a choice out of the few options that the Specialists came up for us.
There are a few other helper nodes like Joiner, but in most cases we tried, the power-trio above seems to be sufficient to build pretty sophisticated agents. Give it a whirl and see how it goes.
📱 One more thing
If you're interested in local models, Breadboard now supports the on-device Gemini Nano model that uses the experimental Prompt API. You have to sign up for early access here, but once you do, you should be able to play with it. Look for it in the Gemini API Kit:
Whew. That's all for now. Until next time. Keep on bakin'!