Full HRI MVP#456
Merged
Merged
Conversation
Final checks to see if it works consistently needs to be carried out
* Ros2 Port Language Package * Introduction testing * Modifications. fix init before final push * Message filter * Launch file * State machine and setup.py modifications * Fix register face and recognise * Port HRI introduce states to YASMIN * Fix YOLO detections in introduce. * Initial code for pick and place * Bug fixes * Fixes after testing on the robot * Black formatting * HRI Task - SM1-3 fully tested and working on the robot (#443) * Testing HRI SM2 * LLM package with venv * Fixed SM3 * RVIZ debug for HRI testing * Fixed lasr_llm by removing broken llama_cpp_python * Successfull SM2 of HRI * WIP - SM2 to SM3 * WIP: CHECKS * WIP - SM1 -> SM3 Final checks to see if it works consistently needs to be carried out * Ported recieve Object * Temporary detect3d fix * Working SM1-3 of HRI --------- Co-authored-by: Aldrich-Fernandes <aldrich.fernandes05@gmail.com> * New VLM service to be used in HRI (#444) * vlm service * service update * Added back license and readme * Formatted files with black --------- Co-authored-by: rajhirym <rym.rajhi@icloud.com> Co-authored-by: Yara Alkhelaiwi <yaralkhelaiwi@gmail.com> * VLM service for HRI task (#445) * Working VLM Replaced old vqa for describing people with new vlm service * Formatted files with black * Fixed accidental merge errors * WIP - Setting up and Testing HRI task * Fix introduce and remove testing code * WIP - HRI task semi-working SM1-4 works correctly, however during SM4 it doesn't look at the seated guests * Change LLM to Ollama got HRI get anem and drink * Updated Seat Guest (SM3) * WIP - HRI SM4 being refactored * Verbal introduction is correct * Debugging SM4 face person errors * Formatted files with black --------- Co-authored-by: Yara Alkhelaiwi <yara.alkhelaiwi@kcl.ac.uk> Co-authored-by: Yara Alkhelaiwi <yaralkhelaiwi@gmail.com> Co-authored-by: rajhirym <rym.rajhi@icloud.com> Co-authored-by: Yara Alkhelaiwi <Yarakmk> Co-authored-by: Aldrich-Fernandes <aldrich.fernandes05@gmail.com> Co-authored-by: Ma'ayan Armony <maayan.armony@gmail.com>
* HRI task (now SM1-5) (#447) * Ros2 Port Language Package * Introduction testing * Modifications. fix init before final push * Message filter * Launch file * State machine and setup.py modifications * Fix register face and recognise * Port HRI introduce states to YASMIN * Fix YOLO detections in introduce. * Initial code for pick and place * Bug fixes * Fixes after testing on the robot * Black formatting * HRI Task - SM1-3 fully tested and working on the robot (#443) * Testing HRI SM2 * LLM package with venv * Fixed SM3 * RVIZ debug for HRI testing * Fixed lasr_llm by removing broken llama_cpp_python * Successfull SM2 of HRI * WIP - SM2 to SM3 * WIP: CHECKS * WIP - SM1 -> SM3 Final checks to see if it works consistently needs to be carried out * Ported recieve Object * Temporary detect3d fix * Working SM1-3 of HRI --------- Co-authored-by: Aldrich-Fernandes <aldrich.fernandes05@gmail.com> * New VLM service to be used in HRI (#444) * vlm service * service update * Added back license and readme * Formatted files with black --------- Co-authored-by: rajhirym <rym.rajhi@icloud.com> Co-authored-by: Yara Alkhelaiwi <yaralkhelaiwi@gmail.com> * VLM service for HRI task (#445) * Working VLM Replaced old vqa for describing people with new vlm service * Formatted files with black * Fixed accidental merge errors * WIP - Setting up and Testing HRI task * Fix introduce and remove testing code * WIP - HRI task semi-working SM1-4 works correctly, however during SM4 it doesn't look at the seated guests * Change LLM to Ollama got HRI get anem and drink * Updated Seat Guest (SM3) * WIP - HRI SM4 being refactored * Verbal introduction is correct * Debugging SM4 face person errors * Formatted files with black --------- Co-authored-by: Yara Alkhelaiwi <yara.alkhelaiwi@kcl.ac.uk> Co-authored-by: Yara Alkhelaiwi <yaralkhelaiwi@gmail.com> Co-authored-by: rajhirym <rym.rajhi@icloud.com> Co-authored-by: Yara Alkhelaiwi <Yarakmk> Co-authored-by: Aldrich-Fernandes <aldrich.fernandes05@gmail.com> Co-authored-by: Ma'ayan Armony <maayan.armony@gmail.com> * Added REID fallback Also made minor changes to accomodate gpu * Test HRI on GPU * Updated HRI to say attributes Also added in a SafeGoToLocation and moved StartDoorSM to skills * Debugging HRI task * Added gpu requirements for lasr_vision_reid * Testing of HRI with GPU * Fixing bugs * Fixing of EyeTracker * Working eye tracker, yolo is bottleneck * Fully fixed eye tracker * Adding in host * Fixed introduce for host * Formatted files * Merge branch 'ros2-hri-sm6' into ros2-gpu-and-reid
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.