XR Open Source Fellowship Program
BUILDING DIGITAL PUBLIC GOODS FOR THE METAVERSE
Basic Information
No. | Details | Response |
1 | Name | Akshay Sharma |
2 | Akshaysharmabitd@gmail.com | |
3 | Project Name / Organisation Name | Parallax Labs |
4 | Project Mentor Name | Rahul Shaw & Satyam Kashyap |
5 | Project Mentor Email | |
6 | Project Title | Angioplasty VR Simulator - Medical Simulator |
7 | Project GitHub Link | |
8 | Number of Hours you worked on during the XROS Fellowship Journey | 30 Hours/Week |
9 | Were the weekly / fortnightly meetings attended by you (Yes / No) | Yes |
Background - |
The journey of the XROS Fellowship (How did you got selected, talk about the journey, talk about the project proposal, interview process etc) ( 200 -250 words) The journey of the XROS Fellowship has been an exciting and fulfilling experience for me as an XR developer and artist. I first came across the fellowship on LinkedIn and was immediately drawn to it, given my passion for extended reality and my participation in various national and international hackathons and competitions focused on AR and VR applications. After learning more about the fellowship and its support from organisations like FICCI, NeGD, Meta, and Reskilll, I decided to apply. I had some doubts initially, but I got them cleared by asking questions in the Slack channel and watching the AMA project explanation video. I shortlisted four organisations for the fellowship, but eventually applied to three, out of which I received interview calls from two. Among the projects available, I was most interested in the one offered by Parallax Labs, which involved developing a medical simulator. The interview with Rahul from Parallax Labs went smoothly, despite me being out of station at that time. I confidently discussed my experience, previous projects, and showcased my portfolio during the interview. The process was rigorous, but I was overjoyed when I received the news of my selection in the XROS Fellowship Program with Parallax Labs on March 31 at 4:28 PM. I loved working on the medical simulator project, utilising tools like Unity Engine, Blender, C#, and Figma to create a realistic and immersive learning experience for healthcare professionals. The opportunity to make a positive impact on the healthcare field through cutting-edge technology is truly rewarding, and I am grateful for the XROS Fellowship for providing me with this platform to grow and contribute to the XR community. |
The Angioplasty VR Simulator is an application designed for users who wish to learn, understand, and familiarize themselves with the angioplasty procedure through immersive virtual reality. Whether the user is a medical student or an aspiring surgeon, they can access this application at any time to receive training on performing angioplasty procedures. The project is currently compatible with PC-based virtual reality systems such as Oculus Quest and HTC Vive, allowing users to connect their head-mounted display (HMD) to a computer either wired or wirelessly. The Angioplasty VR Simulator aims to provide a lifelike and engaging experience to users. By utilizing virtual reality technology, this application allows users to immerse themselves in a realistic virtual environment, replicating the angioplasty procedure. Through interactive simulations, users can actively participate in the procedure, gaining hands-on experience and improving their understanding of the process. The primary focus of the Angioplasty VR Simulator is to provide an accurate representation of the angioplasty procedure. Users can learn about the steps involved in inserting a catheter, navigating it through blood vessels, locating blockages or narrowings, and performing balloon angioplasty. This VR application serves as a valuable educational tool, supplementing traditional training methods for medical professionals. It enables users to practice their skills, develop their hand-eye coordination, and enhance their decision-making abilities in a risk-free environment. The simulator can also provide detailed visualizations and interactive explanations to further augment the learning process. It is worth noting that the Angioplasty VR Simulator is not intended to replace real-world training or serve as a substitute for supervised medical procedures. It is meant to complement existing educational resources, providing an additional avenue for learners to gain practical knowledge and experience in a controlled and immersive virtual environment.
Tech stack used: My contribution in the project:
|
Goals (that you were to achieve) - Top (5-10) goals that were chased and achieved during the XROS Fellowship journey | |
Enhancing Realism: I focused on refining the medical simulator to closely resemble real-world angioplasty procedures. By meticulously fine-tuning the simulation, users can gain a highly realistic experience that accurately mirrors the intricacies of the procedure. | Integrating Arteries into the 3D Model: I successfully incorporated detailed artery models into the overall patient 3D model. This integration allows users to visualize and interact with the arteries during the angioplasty simulation, providing a more comprehensive learning experience. |
Animated Heart Functionality: I implemented a heart animation that synchronizes with the heartbeat per minute. This dynamic animation brings the heart to life, showcasing its movements in sync with the user's input or a predefined rhythm, contributing to a more immersive training environment. | GuideWire and Catheter Insertion: I developed the functionality to guide the wire and catheter from the hand to the heart within the simulation. This involved accurately replicating the procedural steps, allowing users to practice the precise technique of navigating the wire and catheter through the arteries. |
Camera Setup and Monitoring: I successfully configured the camera within the simulation to provide a view of the heart and the insertion of the wire. This feature enables users to observe the wire's movement and progress within the arteries, enhancing their understanding and procedural awareness. | VR-Compatible Shader: I adapted the shader used in the application to ensure compatibility with virtual reality platforms. By optimizing the shader, I ensured that the visuals are properly rendered in the VR environment, maintaining a high level of detail and realism. |
User Interface Design: I designed and implemented a user-friendly interface that guides users through each step of the procedure. This intuitive interface allows users to quickly navigate between menus and the main simulation scene, enhancing their overall learning experience. | High-Quality Equipment 3D Models: I created high-quality and realistic 3D models of the equipment used in angioplasty, drawing references from real-world medical instruments. These detailed models provide users with an accurate representation of the tools they would encounter in a clinical setting. |
How did you go about achieving those goals? |
Take us on a journey with a minimum of 500 words, showcasing the sheer determination and Herculean effort you poured into getting things done! Achieving the goals for the medical simulator project required significant effort and determination on my part. Coming from a non-medical background, I recognized the importance of thorough research before diving into the project. To gain a better understanding of the angiogram and angioplasty surgery procedures, I watched over 10 videos and extensively read research papers and articles available on the internet. This allowed me to familiarize myself with the intricacies of the procedures and ensured that I approached the project with the necessary knowledge. Under the guidance of my mentor, Rahul, it was crucial to create a storyboard that outlined the flow of the simulator from start to finish. I meticulously crafted a 15-step storyboard, taking into account the expectations for high-quality graphics. However, we encountered a setback when exporting the project using the HDRP pipeline. Although the graphics were exceptional within the Unity Editor, the export process proved to be time-consuming. Despite the disappointment, our determination remained steadfast, and we decided to switch to the Universal Render Pipeline (URP). While URP offered slightly lower graphics capabilities than HDRP, we successfully exported the project, ensuring its usability for potential users. One of the unique challenges we faced was creating the guide wire, which would simulate the movement from the patient's hand to the coronary arteries in the heart. As no existing solutions were available, we had to devise our own approach. Extensive research and brainstorming sessions were conducted to develop an innovative solution that met our project's requirements. The fellowship coincided with my final semester of engineering, which posed additional challenges in managing my time effectively. Despite a demanding academic schedule, I diligently allocated dedicated time slots to work on the project while juggling class tests and end-semester exams. With unwavering commitment, I made significant contributions to the project, thanks in part to the unwavering support of my mentors. To further complicate matters, I found myself transitioning between a hostel and my home during the fellowship. This required adaptability and the ability to maintain focus regardless of the environment. Whether working from the hostel or my home, I remained dedicated to the project and ensured its progress remained on track. Throughout the journey, I continually refined and improved my work. Feedback from mentors and fellow participants was invaluable, allowing me to identify areas for enhancement and implement necessary adjustments. I approached each challenge with determination, exploring alternative solutions and seeking guidance when necessary. This Herculean effort and tireless dedication ultimately led to the successful development of the medical simulator, bringing it closer to the realistic and immersive experience we envisioned. In conclusion, achieving the goals for the medical simulator project required a combination of thorough research, adaptability, and sheer determination. From gaining a comprehensive understanding of medical procedures to overcoming technical hurdles, I poured my energy into every aspect of the project. Balancing academic commitments and work, I ensured that progress was made consistently. The unwavering support of my mentors, combined with my relentless pursuit of excellence, allowed me to deliver a high-quality project that aligns with the vision of providing healthcare professionals with a realistic and immersive learning experience. |
Let's discuss work management in detail! | |
Share with us the intricacies of your daily task routine. Give us a step-by-step account of how you flawlessly navigated through your responsibilities day in and day out (Min. 500 words) Flawlessly navigating through daily tasks and responsibilities requires meticulous work management and efficient organization. Here is a step-by-step account of how I managed my workload and responsibilities, ensuring a seamless workflow and successful completion of tasks. | |
Task Tracking and Planning: To effectively manage my daily tasks, I utilized Trello, a project management tool. I set up a Trello dashboard where I could track and prioritize tasks. Each task was assigned a deadline, ensuring a clear timeline for completion. This allowed me to have a comprehensive overview of my responsibilities and prioritize them accordingly. | Project Setup and Model Conversion: In the initial stages of the project, my mentor provided an operating room 3D model in the Unreal Engine format. However, as we decided to work with the Unity Engine, I faced the challenge of converting the model to a compatible format. With the guidance of my mentor, I manually assigned textures to the model to ensure its seamless integration into the Unity Engine. This required attention to detail and a thorough understanding of the model's specifications. |
Collaboration and Communication: Regular communication and collaboration with my mentor were essential for the successful execution of tasks. We initially scheduled one meeting per week to discuss project progress, address any challenges, and align our goals. However, as the program approached its final stages, we increased the frequency of meetings to twice a week. During these meetings, I provided updates on task progress, showcased prototypes, and sought feedback from my mentor. This open line of communication enabled effective collaboration and ensured that I was on the right track. | 3D Modeling: Alongside the provided 3D model, I took on the responsibility of creating models for various equipment and items in the medical simulator. To accomplish this, I referred to real-life images of the equipment and meticulously modeled them using Blender. This involved a combination of technical skill, creativity, and attention to detail to accurately represent the objects in a 3D space. |
Time Management and Adaptability: Throughout the fellowship, I had to effectively manage my time to balance my responsibilities. This included juggling academic commitments, personal life, and the fellowship project. By carefully planning and prioritizing tasks, I ensured that I dedicated focused time to the project, even amidst a busy schedule. Additionally, I had to adapt to different work environments, such as working from a hostel or home, and maintain focus and productivity regardless of the setting. | Task Execution and Feedback Loop: Once I completed assigned tasks, I shared the application or prototypes with my mentor for testing and review. This allowed my mentor to identify any bugs or areas of improvement that may have been overlooked during my own testing. I carefully noted the feedback provided and promptly addressed the identified issues. This iterative process of receiving feedback, making necessary adjustments, and seeking further guidance from my mentor ensured continuous improvement and refinement of the application. |
Commitment to Quality: A key aspect of flawlessly navigating through daily tasks was maintaining a high standard of quality in my work. Whether it was modeling 3D objects or coding functionalities, I paid meticulous attention to detail and strived for excellence in every task. This commitment to quality ensured that the final deliverables met the desired standards and exceeded expectations. In conclusion, flawlessly managing daily tasks and responsibilities required effective task tracking, collaboration, and communication with my mentor. By utilizing project management tools, maintaining a feedback loop, and managing my time effectively, I successfully executed my responsibilities. With attention to detail, adaptability, and a commitment to quality, I navigated through the intricacies of the project, overcoming challenges and consistently making progress toward project goals. | |
Pull Requests and Issues that you contributed to -
Share all the contributions you made! Provide us with a comprehensive list of the Pull Requests and Issues you worked on. Include all the relevant links to showcase your amazing collaboration. Get ready to impress us with your involvement and the valuable impact you made!
Each pull request encompasses numerous updates, new features, and addresses specific issues encountered during the fellowship.
| #1 Pull Request |
| |
| |||
Youtube dev-log as of May 21, 2023 - https://youtu.be/ieXU9tTEsFM | |||
| #2 Pull Request | | |
| |||
Youtube dev-log as of June 21, 2023 - https://www.youtube.com/watch?v=7cnBMVevJno | |||
| #3 Pull Request |
| |||||||||
| |||||||||||
| #4 Pull Request |
| |
| |||
| #5 Pull Request |
| |
| |||
Youtube dev-log as of July 17, 2023 - https://www.youtube.com/watch?v=jnjg8VtLi0Q&t | |||
Your suggestions for the project owners specifically for the project that you worked upon (list down all the suggestions in a numerical manner) | |
The following suggestions are not only for project owners but also for anyone who would like to contribute to this open-source project: | |
Enhance Realism: Incorporate more realistic physics and interactions to further enhance the immersion and authenticity of the medical simulator. This can include improving the haptic feedback and responsiveness when handling medical instruments or objects within the virtual environment. | Expand Procedure Options: Consider adding additional medical procedures to the simulator to provide a broader learning experience for healthcare professionals. This could involve simulating different types of surgeries, interventions, or diagnostic procedures, allowing users to practice a wider range of skills and scenarios. |
Incorporate Dynamic Scenarios: Introduce dynamic and unpredictable scenarios within the simulator to simulate real-world challenges that healthcare professionals may encounter. This can include emergency situations, complications, or variations in patient conditions, requiring users to adapt their skills and decision-making abilities in a dynamic environment. | Integrate Feedback System: Implement a feedback system within the simulator that provides users with constructive feedback on their performance, highlighting areas of improvement and best practices. This can help users track their progress, identify areas for growth, and promote continuous learning and development. |
Collaborative Learning Opportunities: Explore the possibility of incorporating collaborative learning features into the simulator, allowing multiple users to engage in teamwork and communication exercises. This can simulate interdisciplinary teamwork scenarios, fostering collaboration and effective communication skills among healthcare professionals. | Mobile Compatibility: Consider developing a mobile version of the simulator to expand accessibility and reach. This would enable healthcare professionals to access the training and learning experience on their smartphones or tablets, making it more convenient for on-the-go learning and practice. |
Multi-platform Support: Explore the option of extending the simulator's compatibility to different virtual reality platforms, such as Oculus Rift, HTC Vive, or Windows Mixed Reality headsets. This would ensure that healthcare professionals can access and benefit from the simulator regardless of the VR platform they have access to. | Analytics and Progress Tracking: Implement an analytics system within the simulator to track user progress, performance metrics, and skill development over time. This data can be used to provide personalized recommendations, tailor learning experiences, and assess the effectiveness of the training program. |
Continuous Content Updates: Regularly update and expand the content and scenarios within the simulator to ensure that it remains up-to-date with the latest advancements in medical procedures and technologies. This can include adding new equipment, updated guidelines, or incorporating emerging trends in the healthcare field. | User Feedback and Iterative Development: Foster a feedback loop with users, including healthcare professionals and trainees, to gather insights and suggestions for further improvement. Actively listen to user feedback and iterate on the simulator based on their recommendations, ensuring that it continues to meet the evolving needs of the target audience. |
These suggestions aim to enhance the project's impact, user experience, and educational value, ultimately providing a valuable training resource for healthcare professionals and contributing to advancements in the field of medical simulation. | |
Talk about the Mentor and how he / she helped you achieve your daily / weekly / monthly goals | |
My mentor Rahul & Satyam played a crucial role in helping me achieve my daily, weekly, and monthly goals throughout the project. Their guidance, support, and expertise were invaluable in ensuring a successful and productive collaboration. Here's how my mentor contributed to my progress: | |
Clear Communication: My mentor maintained open and clear lines of communication. They were readily available to answer any questions, provide clarification, and offer guidance whenever needed. This allowed me to have a clear understanding of the project objectives, expectations, and tasks at hand. | Goal Setting and Planning: At the beginning of the project, my mentor helped me set clear goals and objectives for each phase and provided guidance on how to break down the project into manageable tasks. This allowed me to create a roadmap and plan my work effectively, ensuring that I stayed on track and met the project milestones. |
Task Allocation and Prioritization: My mentor helped me prioritize tasks based on their importance and deadlines. They provided insights into the critical aspects of the project and guided me in allocating my time and resources efficiently. This ensured that I focused on the most important tasks and made progress according to the project timeline. | Expertise and Technical Guidance: As an experienced professional in the field, my mentor provided valuable technical guidance. They shared their expertise, best practices, and industry insights, helping me navigate complex challenges, make informed decisions, and implement solutions effectively. Their knowledge and guidance significantly contributed to the quality of my work. |
Feedback and Iterative Improvement: Throughout the project, my mentor provided continuous feedback on my work. They reviewed my code, prototypes, and designs, providing constructive criticism and suggestions for improvement. This feedback loop allowed me to refine my work, address any issues or shortcomings, and enhance the overall quality of the project. | Problem Solving and Troubleshooting: Whenever I encountered technical or conceptual challenges, my mentor was there to assist me. They helped me analyze the problems, explore potential solutions, and brainstorm ideas to overcome obstacles. Their guidance and support empowered me to find effective solutions and learn from the problem-solving process. |
Motivation and Encouragement: My mentor provided constant motivation and encouragement throughout the project. They recognized and appreciated my efforts, celebrated milestones and achievements, and provided encouragement during challenging times. Their support kept me motivated, boosted my confidence, and inspired me to give my best. | Professional Development: In addition to project-specific guidance, my mentor also offered insights and advice for my professional development. They shared career advice, recommended resources for learning, and encouraged me to explore new technologies and techniques. Their guidance went beyond the project, helping me grow as a professional in the field. |
In conclusion, my mentor played a crucial role in my project's success by providing guidance, support, expertise, and motivation. Their clear communication, goal setting, technical guidance, feedback, and encouragement significantly contributed to achieving my daily, weekly, and monthly goals. Working with a mentor of such caliber enhanced my learning experience and helped me develop valuable skills and knowledge in the process. I am grateful for their invaluable contributions to my growth and the successful completion of the project. | |
Special Mention - you can mention / talk about your faculty members, friends etc if they helped you in cracking XROS fellowship or in any other manner whatsoever. |
During the IT Day Hackathon in Jaipur, Rajasthan, I received an email to schedule an interview for the XROS Fellowship using Calendly. As I made my way to Udaipur, my friend was accompanying me on the journey. The interview was scheduled for the evening, and my friend provided the motivation and encouragement to proceed with the interview instead of canceling or postponing it, recognizing the tremendous opportunity it represented. Having back-to-back interviews scheduled on that day, attending both was my priority. However, I faced a challenge when I discovered that the hotel's Wi-Fi in Udaipur was unreliable. Determined to overcome this obstacle, I quickly resorted to using my phone's hotspot for a stable internet connection. Unfortunately, just before the final interview, my data was exhausted. In a moment of urgency, I reached out to my friend for assistance. Understanding the significance of the opportunity, my friend, who was also in Udaipur with me, responded swiftly to my request. Within 10 minutes, they returned to the hotel and provided me with their hotspot connection. Their support and willingness to help ensured that I could participate in the interview without any interruptions. I am deeply grateful for their prompt assistance. Thanks to my friend's timely intervention, I was able to successfully navigate the interviews and make the most of the XROS Fellowship opportunity. Their unwavering support and the provision of their hotspot connection exemplified their commitment to my success. I am truly fortunate to have such a dedicated and supportive friend by my side. |
Post a Comment