ContentEd AI in Practice in 11 minutes: how good is our AI summary?
Last week we hosted the ContentEd AI in Practice conference (25-26 March 2025). Our online conference took place over two half-day sessions, bringing together participants from around the world in a discussion about the practical applications of AI in education marketing and communications.
It wouldn’t be an AI conference if we didn’t actually use AI in the conference production itself. In addition to using generative AI to help us create and edit promotional content for the event, we also used it to experiment with creating conference summaries. Here’s how that turned out… (run time: 11 minutes 28 seconds)
So, we used NotebookLM Plus (one of Google’s AI studio products) to create this. It’s all pretty simple and is designed to help you summarise large volumes of content.
We uploaded audio files of all of the conference talks (except our panel discussion) into NotebookLM Plus. This was around 5 hours of recordings in total.
We provided a very brief instruction on what to focus on (this is limited to 500 characters, so you have to be brief)
We iterated on that brief a few times, eventually getting to a summary that was reasonably short and fairly reliable (others were close to 30 minutes).
So, how did it do?
Overall it captured the essence of the talks pretty well. It didn’t tend to make anything up (hallucinate) but it did slightly misinterpret a few things (for example it says that the University of Sussex chatbot is called Dante, but actually it’s called USAID and the tool that we at Pickle Jar used to create it for them is called Dante). But mostly those misinterpretations were understandable and something that a human could equally mistake too.
The quality of the audio is pretty good, with very few glitches or evidence that it’s a totally AI generated conversation.
It struggles with some pronunciation. It mispronounces a few speaker last names, and it struggles with “ContentEd” too (but then so do humans!). However, in our brief we specifically told it how to pronounce ContentEd correctly. It actually got it correct in some earlier iterations but in the final short version it fell short.
The main limitations of using NotebookLM in a use case like this that you want to share publicly and not just save for personal use are:
It only works in English (and the presenters only have North American accents - but we’ll forgive that!)
You can only provide 500 characters of instructions so it’s not easy to brief it thoroughly enough to really influence the output
You can’t iterate on a result. Your only option if you don’t like what it created is to delete it and start over.
So, for now it’s probably great for personal use and helping you to distill what’s in large volumes of content when you’re time poor, but the lack of control or ability to prompt it well makes it less reliable for something you’d use for external use.
Interacting with the audio
Another cool feature that’s currently in beta is the interactive mode. This mode allows you to interact and ask questions of the presenters. Basically you listen to them and interrupt them by clicking on “join”. You ask a question and they attempt to answer it. I found this to be more glitchy, with definite delays on responses but, heck, it’s in beta mode so I don’t expect perfect. Have a listen to me playing with this (run time: 2 minutes 38 seconds):
A screen shot of how the interactive feature appears in NotebookLM. It’s as simple as hitting the “join” button and asking your question with your voice (you need your mic enabled in your browser first, of course).
Want to watch the whole conference and learn more about practical applications of AI in education?
Listening to an 11 minute summary isn’t as good as attending the whole thing. And as the conference host, I can tell you these sessions were brilliant and so inspiring. But maybe the summary piqued your interest? You can purchase access to watch the whole conference playback on demand over on the ContentEd website.