Hongfei Ji
中文

Playground+

Design Technology Research

Project Source

Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems

Collaborators

Gerogie Qiao Jin

Project Year

2026

Physical activity play (PAP) is vital for children's physical, cognitive, and social development, and previous systems and research have explored how various technologeis can support it. However, existing approaches often offer limited game choices, rely heavily on family initiative for ideation, and struggle to adapt to everyday PAP practices in everyday settings. AI-enabled mixed reality (AI-MR) systems offer new opportunities through their ability to perceive physical context, support play ideation, and lower barriers for lay families to create PAP. We present Playground+, a projection-based AI-MR system that combines computer vision and large language models to ideate, generate, and guide PAP. Using a ceiling-mounted projector and webcam, the system provides multimodal, wearable-free guidance that accommodates different parental roles and children's developmental levels. Future work will focus on designing temporal and social support, comparing with spatially flexible solutions and evaluating self-expression support through user studies, and addressing technical and deployment challenges.
The projects overview

BACKGROUND

  • Physical activity play is a form of play centered on engagement, bodily coordination, and spatial interaction, and is important for children’s development.
  • Existing systems often provide limited choices and fixed formats, thus families still need to create and adapt play in everyday home settings.
  • We present Playground+ to explore how projection-based AI–MR can lower this barrier by generating and guiding open-ended, wearable-free family play in domestic environment.
Problem Statement: Existing systems offer important benefits, but they often lack flexibility and support for everyday adaptation, creating a high barrier to creative play.

PLAYGROUND+

Playground+ is designed to be easily deployable at home using a ceiling-mounted home projector (JMGO P5X) 3 and a webcam (UGREEN CM827). We used an adjustable photography stand sized to the room and temporarily mounted both devices on the stand for testing. The devices were powered and connected via cables routed along the walls and stand, and connected to a personal laptop that runs the system.

To address design goals, the system consists of three components:

  1. a multimodal conversational agent that perceives spatial states and interprets family preferences;

  2. a reasoning agent that uses a back-end AI model to produce diverse, detailed, and executable play plans per request

  3. a runtime engine that renders visual cues in-situ for family players and supports monitoring of safety, physical states, and play intentions.

Playground+ System Overview

To use the system, families install the devices above a designated play zone (e.g., a cleared area in the living room or a children’s play mat) and bring selected household objects or toys to this space. Once a participant enters the space, the play session begins. The system first asks family members to communicate their needs by answering four predefined questions about personal information (name and age), play duration, play preferences, and desired constraints. It may request additional details until sufficient context is collected to design a PAP session. Then, the system generates the play activity and prompts the family to prepare or take a short break while processing. When the instructions are ready, family members follow the projected guidance on the floor to complete the round. After each round, families can replay the same activity without changes, regenerate the activity with modified rules or preferences, start a new session from scratch, or stop and exit.

Example outputs of Playground+: (a) Obstacle Race, where household objects are used as obstacles; (b) Hopscotch, generated as request when no objects are present in the scene; (c) Toy Toss, ideated and generated by the system when the user shows no preference.

DISCUSSION AND FUTURE WORK

  • Examine the tension between structured guidance and open-ended play
  • Extend from spatial guidance to temporal and social support for family engagement, physical states, and personalization
  • Compare projection-based play with more flexible AI-supported approaches
  • Improve robustness for home deployment through better sensing accuracy, reduced occlusion/perspective issues, and simpler hardware setup

FULL PAPER

← Back to Projects