Integrating voice and speech recognition systems into daily personal use is a recent technological shift. While these tools have become accessible to a broader audience, their experimentation lacks a focus on affective feedback and creative expression. Our project, kaleidOk, recognized this gap and aimed to explore the potential of creative speech, storytelling, poetry, singing, and other meaningful word use with computer systems.

 

KaleidOk is uniquely focused on visually communicating emotions derived from spoken words, encouraging users to express themselves in public screen-mediated interactions. The features of kaleidOk included computing emotion from speech, retrieving images based on emotional input, and artistically displaying them with vocal attributes. The design, informed by empirical considerations and subjective criteria, sought to bridge the gap between technology and human emotional expression.

 

User testing played a crucial role in shaping the evolution of kaleidOk. The findings indicated that the most effective use case involved a more intimate screen interaction experience, leading to the decision to continue kaleidOk as a console-based setup in public spaces. Deployed in diverse contexts, from hackathons to cafés, cinemas, universities, and public spaces, kaleidOk encountered challenges concerning public speaking apprehension and expressing emotions in a public setting. Despite these challenges, user responses highlighted kaleidOk as an innovative concept that is engaging, aesthetically pleasing, reflective of users' emotional input, and instrumental in making users more aware of their current emotional state.

 

In our ongoing work, we aim to address several key aspects. Firstly, we plan to explore emotion detection from the voice by incorporating alternative speech-to-text services, such as CMU Sphinx. Additionally, we aim to enhance kaleidOk by analyzing tonal features to contribute to color search and texture generation, expanding the scope to include singing and other vocal expressions. We also aspire to incorporate an alternative image search index that is accessible and contributable to local community audiences, fostering community engagement.

 

Future user studies will delve into user experience and interface design, considering the unique challenges of public speaking and emotional expression in public settings. Moreover, our research will focus on developing methodologies to test emotional accuracy, ensuring reproducible outcomes, and refining the technology's effectiveness in capturing and reflecting users' emotional states.

 

kaleidOk represents an exciting venture into the uncharted territory of creative speech and emotional expression through technology. As we continue our exploration, we are committed to refining and expanding kaleidOk's capabilities, addressing user concerns, and pushing the boundaries of technology's role in facilitating meaningful human expression.