Project artifact · 2023 · archived
FrankTTS → AonyxBuddy: a companion takes shape
A new follower arrives. A raid brings a roomful of people. A familiar name appears in chat. I built the original AonyxBuddy to give those moments a voice: a small on-screen companion with responses, a personality, and a mouth that moved as it spoke.
This began as FrankTTS, some of my earliest work on tools for content creators. Its surviving Git history reaches back to June 2023. It started with a character tailored to one stream and developed toward AonyxBuddy: a shared client with different voices, artwork, greetings, and channel settings. The original repository and later archived versions capture that progress.
From the original repository
FrankTTS is the public starting point for this story. The first surviving commit, on 17 June 2023, already calls the project version 0.0.3. The JavaScript source and Webpack build show how the early overlay was assembled, with configuration bundled into the application.
The develop branch goes further: late-June work introduces asynchronous configuration loading and VTube Studio integration. The repository’s later archive note identifies FrankTTS as the original version of AonyxBuddy and AonyxBuddyVT.
The examples below also draw on later AonyxBuddy files recovered from my old server. They show where the idea went; they are not all builds of the current FrankTTS repository.
Meet the original companions
Original sprites · recreated preview
SolOriginal voice: Joey
visitor follows the channel
“Thanks for following, visitor! Lets build a community together!”
Play a regenerated Joey greeting. These saved demo clips drive the mouth animation; reduced motion keeps it still. You can also explore the frames with the slider.
The idea: a companion that could participate
Stream alerts can announce that something happened. AonyxBuddy explored what it would feel like for a character to respond to it.
The plugin listened for follows, subscriptions, gifted subs, cheers, raids, and chat through StreamElements. It selected a response, filled in details such as a viewer’s nickname or the size of a raid, and queued the text for speech. First-chat greetings helped the character acknowledge people arriving in the stream.
Moderators and the streamer could also direct it with chat commands: ask it to say something, skip a message, clear pending speech, or mute and unmute. That made the companion both a performance element and something the streamer could control during a live broadcast.
What I built
- An event-to-response pipeline. Different Twitch events became a common format, then passed through filters, permissions, nickname substitutions, and response selection.
- Speech with a visible presence. StreamElements supplied the speech audio; Web Audio measured its amplitude and selected a talking frame. PNG frames and GIF animation gave the character an expressive, lightweight appearance.
- Configurable personalities. Sol, Frank, and Chuck shared much of the same machinery while using different voices, sprite sets, response writing, and commands.
- A remotely loaded client. A shared loader fetched the channel’s bundle, bridged WebSocket events into the page, and supported a refresh message. It was an early step toward updating the experience centrally.
The important connection was between these pieces: a chat event could become a personalized sentence, then speech, then movement on screen.
Progress inside the early versions
01 — A character built for a stream
The FrankTTS source selected a user configuration and combined event listeners, filters, a speech queue, and a canvas renderer. Some choices were fixed in code, including the Brian voice and a 256 × 256 canvas. The oldest recovered bundle retains that structure. It already connected the core loop from stream activity to a speaking character.
02 — The personality becomes configuration
Later versions exposed voice, canvas dimensions, sprite references, command groups, nicknames, and response templates through configuration. Sol used Joey’s voice and a sun-like avatar; Frank used Brian and a small creature; Chuck used Matthew with a different canvas size and artwork.
This was a meaningful change in scope. Making another companion increasingly meant describing its personality and appearance rather than rewriting its behavior.
03 — Iterating on the shared behavior
The recovered builds also show concrete corrections. Frank adds filtering for cheer tokens and fixes a condition that caused Sol to enqueue ordinary event messages. Two Chuck versions have identical runtime code but different owner and nickname settings—a small, clear example of configuration doing its job.
These are snapshots of work in progress, with both reused code and changes to the underlying system.
What the idea could have grown into
At the time, the promising direction was a companion toolkit: one set of behaviors, many characters, and a way for streamers to make those characters feel at home in their own communities.
The existing configuration offered the beginnings of that toolkit. An editor could have made personalities easier to author. More event adapters could have connected the same character to other sources. Stronger queue handling and recovery could have made it more dependable during a live stream. Those are extensions suggested by the architecture, not features I’m claiming these versions shipped.
Looking back at the work
Recovering the original code makes the development visible. The early versions contain real rough edges: skipped speech is removed in an unexpected order in the later queue, failed audio requests can interrupt progression, and some configured event types never received a complete implementation.
What I want to preserve here is the trajectory: from a custom character toward reusable event handling, separate configuration, and a more adaptable creative tool. It is an early chapter in my work on AonyxBuddy, and a useful record of how an idea starts becoming a system.
Based on the FrankTTS repository, its development branch, and later bundles recovered from my old server. The preview recreates their interaction concept; it is not a live instance of the old service.