September 2024 ~ November 2024
🏆 과학기술정보통신부 주최 XR 디바이스 콘텐츠 메이커톤 2위 (정보통신산업진흥원장상)
Introduction
i-There is a deep learning-based ADHD screening solution for children developed during the 'XR Device Content Makerthon 2024' hosted by the Ministry of Science and ICT, spanning from the summer break to the second semester of my junior year. This competition was a service development contest based on 'MERALENSE2', a next-generation XR glass developed through a strategic partnership between P&C Solution, a domestic XR hardware specialist, and Qualcomm. The competition was open to startups, research institutes, and the general public without restrictions. After a preliminary review, 10 finalist teams were selected to develop products using the SDK of actual industrial equipment over approximately 40 days. I conceptualized and pitched the idea within a university alliance club to form the team. As team leader, I led a seven-member team comprising two developers, three designers, and one planner, establishing a role distribution and collaboration system that enabled members from diverse academic backgrounds—engineering, design, medicine, and content creation—to maximize their respective expertise. Despite the physical constraints of team members being distributed across different universities and regions, I led the entire process from setting clear objectives for competition participation to role allocation and product development, establishing an efficient communication system. As a result, we achieved a high-quality outcome, winning second place (Korea IT Industry Promotion Agency Director's Award) and a prize of 5 million KRW in this nationwide competition. Building on this project's achievements, I later extended the research by participating in the CHI 2025 Student Design Competition at an international academic conference.
BACKGROUND
Children with ADHD often experience negative self-identity and emotions during psychiatric diagnosis and treatment processes. While physiological approaches such as biomarker discovery, brain imaging analysis, neurotransmitter research, and genetic testing are actively advancing in psychiatry, there remains a relative deficiency in carefully understanding the psychological background of children visiting psychiatric clinics and providing solutions based on this understanding. This project defined this issue as a core challenge and conducted user-centered research.
ADHD is a condition requiring early detection and prompt intervention. Of children visiting psychiatric outpatient clinics, 46.7% are diagnosed with ADHD, with approximately 60% experiencing symptoms into adulthood. These children face heightened risks of academic underachievement, behavioral issues, and interpersonal difficulties, making early diagnosis and treatment essential. However, the diagnostic and pharmacological treatment process imposes significant psychological burden. Prior research indicates that many children develop self-blame, negative self-identity, and guilt toward their parents. These negative emotions can escalate into secondary mental health issues—depression, anxiety disorders, antisocial personality disorder—creating a vicious cycle that undermines treatment efficacy. Therefore, effective ADHD treatment demands an integrated approach combining medical intervention with psychological support that alleviates guilt and fosters positive self-identity formation.
The Ministry of Health and Welfare is advancing the integration of digital technology in mental health to establish objective data frameworks. Currently, psychiatric diagnosis relies heavily on subjective information such as "I feel depressed" or "I hear voices," highlighting the need for a personalized system where patients can collect digitalized, objective measurements—comparable to blood pressure or glucose levels—and collaborate with physicians in determining treatment approaches. In this context, Digital Therapeutics (DTx) has gained attention for its capacity to quantify and objectively measure symptoms through data-driven approaches, thereby complementing the limitations of subjective diagnosis and enabling personalized treatment based on individual data. DTx has demonstrated significant efficacy in ADHD treatment, with clinical guidelines from the American Academy of Pediatrics (AAP, 2019) and the UK's National Institute for Health and Care Excellence (NICE, 2018) recommending behavioral therapy or parent training programs alongside pharmacological treatment for children and adolescents aged 6-18 with ADHD.
This project specifically targets children aged 8-12, a critical period for self-identity formation. This age range falls within the 6-18 years recommended for behavioral therapy by the American Academy of Pediatrics (AAP) clinical guidelines and aligns with the target demographic of EndeavorRx, an FDA-approved ADHD treatment device. The system employs child-friendly user experience design, generating personalized screening data through XR datasets and deep learning analysis. Based on this data, it delivers customized mixed reality programs designed to effectively alleviate the psychological burden children experience during ADHD diagnosis and treatment processes.
DESIGN
The system collects and analyzes multi-dimensional behavioral data in real-time within an immersive virtual environment using the MERALENSE2 XR glasses. Child-friendly storytelling and visual design facilitate natural engagement, while embedded sensors capture hand gesture patterns through hand tracking and head movements via IMU sensors to measure physical hyperactivity and attention dispersion. A voice recognition API analyzes verbal impulsivity indicators including high-frequency patterns, volume, and premature response initiation before question completion. Upon task completion, the system records task performance time, interim disengagement duration, and task-switching intervals in an integrated manner. Collected raw data is efficiently managed in memory through Unity engine's static arrays, undergoes normalization and filtering preprocessing, and is fed into a Barracuda-based deep learning model. Diagnostic results derived through real-time AI inference, along with raw data, are transmitted to the server via REST API for permanent storage in the database, serving as a dataset for continuous model refinement and longitudinal symptom tracking.
The 18 symptom criteria from DSM-5 ADHD diagnostic standards were converted into measurable digital indicators within a virtual environment based on evidence from prior research. A deep learning model was employed to precisely classify and quantify ADHD symptoms manifested through multi-dimensional data including wrist movements, gaze tracking, vocal patterns, and task performance times. Convolutional Neural Networks (CNNs) learn meaningful patterns from time-series behavioral data analogous to extracting morphological features from images. For instance, the model captures characteristic signals of repetitive unnecessary wrist movements during task performance or moments of concentration decline, while filtering out individual baseline activity level variations to extract only behavior features directly associated with ADHD symptoms. This is achieved through multi-layer convolutional architectures that learn hierarchical representations from low-level features (e.g., momentary movement changes) in raw data to high-level features (e.g., sustained hyperactivity patterns). The AI model, integrated with Unity through Barracuda, analyzes collected data in real-time to evaluate each symptom on a 0-3 scale and calculates a total severity score out of 54 points in K-ARS-IV format.
The accumulated diagnostic database can be extended into a personalized ADHD improvement assistance program. Similar to MORA's model—where rehabilitation exercise therapy is prescribed through a medical-exclusive solution and performed via a patient application—this system can provide customized therapeutic content tailored to individual symptom levels, such as adjusting the difficulty of movement control tasks based on hyperactivity severity and modulating attention training task difficulty according to concentration deficiency levels.
Children with ADHD exhibit weak attentional control, lose focus easily, and struggle with simultaneous attention to multiple tasks. To address these cognitive limitations, the system prioritizes intuitive and clear interface design. Reflecting research findings that increased physical distance from objects in AR environments disperses children's attention and consumes greater cognitive energy, the system modulates visual stimulus intensity based on distance while maintaining continuity in color and form. The character was designed as a round, soft-textured clay pig to provide psychological comfort and familiarity, utilizing pastel tones to minimize visual burden. Instructions are delivered slowly and clearly through AI voice, while fairy-tale spatial modeling facilitates natural immersion. Upon mission completion, immediate rewards through 3D effects enable children to experience achievement and sustain engagement throughout the program.
The program begins with the child encountering a baby pig character. Natural immersion is facilitated through dialogue with the baby pig, with an interaction implemented using Naver Clova API where the child introduces themselves and their spoken name is automatically registered. The core tasks are designed to collect data on all 18 symptoms of inattention and hyperactivity-impulsivity from DSM-5 diagnostic criteria. In Step 1, children collect house-building materials in sequence, with total travel distance measuring hyperactivity levels and incorrect material contact frequency assessing inattention. Step 2 presents a task of quietly observing clay baking in an oven, evaluating sustained attention capacity through time spent gazing away from the oven and distractibility through attention shift frequency during character speech. These staged contents induce voluntary participation through game elements and storytelling while simultaneously collecting clinically meaningful behavioral data in an objective and quantitative manner.
Step 3 provides personalized symptom improvement training through a task of stacking bricks according to rules. This stage dynamically adjusts difficulty based on individual symptom levels, with data collected from Steps 1 and 2 input as tensors and evaluated against learned criteria. Children experience practical symptom improvement training through activities requiring sequential organization and systematic execution. In the Final Stage, children who complete all missions finish constructing the baby pig's brick house and receive achievement rewards, concluding the program. These subsequent stages serve as a prototype demonstrating expandability beyond simple diagnosis toward mixed reality digital therapeutics. Through personalized training content based on individual diagnostic data and immediate reward systems, the platform possesses scalability to evolve into an integrated solution encompassing screening through therapeutic intervention.
BUISNESS MODEL
The system establishes a sustainable business model through a three-phase expansion strategy. Phase 1 employs a B2C model, directly providing MERALENSE2 and i-There to children and guardians seeking ADHD symptom improvement, generating initial revenue through device sales and subscription services while accumulating user behavioral data. Phase 2 leverages accumulated data to operate pilot programs with schools and educational institutions, objectively validating ADHD symptom improvement efficacy. This establishes credibility with public agencies and creates a foundation for government support and subsidy eligibility. Phase 3 utilizes validated clinical outcome data to build partnerships with insurance companies and integrate i-There into insurance coverage benefits. By applying a performance-based pricing model that charges only upon demonstrated symptom improvement, the system mitigates insurers' financial risk and establishes long-term strategic partnerships. This approach simultaneously achieves stable revenue structure and market share expansion.
In November 2024, the team participated in the final presentation and demonstration evaluation held at Pangyo Metaverse Hub. The CEO of P&C Solution, developer of MERALENSE2 and serving as a judge, expressed particularly high interest in the project and directly proposed continuing the project with additional provision of the then-incomplete eye-tracking functionality. During the demonstration evaluation, critical questions were raised regarding the validity of AI model usage and data reliability assurance when measurements exceeded device range. The team responded by explaining data filtering methods through sensor fusion algorithms and exception handling logic. In the final results announcement, the team won second place and received the Korea IT Industry Promotion Agency Director's Award along with a prize of 5 million KRW, competing against active industry professionals and master's/doctoral-level research teams.