Papers
Topics
Authors
Recent
Search
2000 character limit reached

Anticipate & Collab: Data-driven Task Anticipation and Knowledge-driven Planning for Human-robot Collaboration

Published 4 Apr 2024 in cs.RO and cs.AI | (2404.03587v1)

Abstract: An agent assisting humans in daily living activities can collaborate more effectively by anticipating upcoming tasks. Data-driven methods represent the state of the art in task anticipation, planning, and related problems, but these methods are resource-hungry and opaque. Our prior work introduced a proof of concept framework that used an LLM to anticipate 3 high-level tasks that served as goals for a classical planning system that computed a sequence of low-level actions for the agent to achieve these goals. This paper describes DaTAPlan, our framework that significantly extends our prior work toward human-robot collaboration. Specifically, DaTAPlan planner computes actions for an agent and a human to collaboratively and jointly achieve the tasks anticipated by the LLM, and the agent automatically adapts to unexpected changes in human action outcomes and preferences. We evaluate DaTAPlan capabilities in a realistic simulation environment, demonstrating accurate task anticipation, effective human-robot collaboration, and the ability to adapt to unexpected changes. Project website: https://dataplan-hrc.github.io

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. R. Arora, S. Singh, K. Swaminathan, A. Datta, S. Banerjee, B. Bhowmick, K. M. Jatavallabhula, M. Sridharan, and M. Krishna, “Anticipate & act : Integrating llms and classical planning for efficient task execution in household environments,” in International Conference on Robotics and Automation, 2024.
  2. M. Ghallab, C. Knoblock, D. Wilkins, A. Barrett, D. Christianson, M. Friedman, C. Kwok, K. Golden, S. Penberthy, D. Smith, Y. Sun, and D. Weld, “Pddl - the planning domain definition language,” 08 1998.
  3. M. Helmert, “The fast downward planning system,” Journal of Artificial Intelligence Research, vol. 26, pp. 191–246, jul 2006.
  4. A. Smith and B. Jones, “Advancements in human-robot interaction for household tasks: A review,” International Journal of Robotics Research, vol. 41, no. 3, pp. 345–367, 2022.
  5. C. Lee and D. Kim, “Collaborative robots: Challenges and opportunities in human-robot interaction,” IEEE Transactions on Robotics, vol. 39, no. 2, pp. 210–228, 2023.
  6. M. Kraus, N. Wagner, W. Minker, and et al., “Kurt: A household assistance robot capable of proactive dialogue,” in Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction, ser. HRI ’22.   IEEE Press, 2022, p. 855–859.
  7. R. Garcia and S. Martinez, “Learning from demonstration for adaptive control in human-robot collaboration,” Journal of Artificial Intelligence Research, vol. 56, pp. 78–95, 2022.
  8. J. Park and H. Kim, “Collaborative task planning for human-robot interaction: Recent advances and future directions,” IEEE Robotics and Automation Magazine, vol. 31, no. 2, pp. 77–94, 2024.
  9. Y. Wang and J. Li, “Context-aware task planning for robot assistance in daily living,” Robotics and Autonomous Systems, vol. 95, pp. 210–225, 2022.
  10. S. Kim and H. Lee, “Human-robot task planning using probabilistic graphical models in smart homes,” ACM Transactions on Intelligent Systems and Technology, vol. 12, no. 3, pp. 45–62, 2021.
  11. Q. Chen and Z. Liu, “Collaborative task planning for household service robots: A review of approaches and challenges,” International Journal of Social Robotics, vol. 16, no. 2, pp. 189–207, 2024.
  12. X. Li and H. Wang, “Learning-based task planning for human-robot collaboration in smart homes,” IEEE Transactions on Robotics, vol. 41, no. 2, pp. 321–338, 2022.
  13. L. Yang and Q. Zhang, “Adaptive task planning and execution for human-robot collaboration in dynamic environments,” Robotics and Autonomous Systems, vol. 85, pp. 150–165, 2021.
  14. W. Liu and J. Huang, “Reinforcement learning for task planning and execution in human-robot interaction,” IEEE Transactions on Cybernetics, vol. 54, no. 1, pp. 78–94, 2024.
  15. Z. Cai, Z. Feng, L. Zhou, C. Ai, H. Shao, and X. Yang, “A framework and algorithm for human-robot collaboration based on multimodal reinforcement learning,” Computational Intelligence and Neuroscience, vol. 2022, pp. 1–13, September 2022.
  16. J. Felip, D. Gonzalez-Aguirre, and L. Nachman, “Intuitive & efficient human-robot collaboration via real-time approximate bayesian inference,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 3093–3099.
  17. E. Brown and F. White, “Addressing safety and trust in human-robot collaboration for household tasks,” Robotics and Autonomous Systems, vol. 87, pp. 123–139, 2021.
  18. K. Valmeekam, A. Olmo, S. Sreedharan, and S. Kambhampati, “Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change),” in NeurIPS 2022 Foundation Models for Decision Making Workshop, 2022.
  19. B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone, “Llm+p: Empowering large language models with optimal planning proficiency,” 2023.
  20. Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, and H. Soh, “Translating natural language to planning goals with large-language models,” 2023.
  21. F. Joublin, A. Ceravola, P. Smirnov, F. Ocker, J. Deigmoeller, A. Belardinelli, C. Wang, S. Hasler, D. Tanneberg, and M. Gienger, “Copal: Corrective planning of robot actions with large language models,” 2023.
  22. K. Valmeekam, M. Marquez, A. Olmo, S. Sreedharan, and S. Kambhampati, “Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change,” 2023.
  23. E. Hirsch, G. Uziel, and A. Anaby-Tavor, “What’s the plan? evaluating and developing planning-aware techniques for llms,” 2024.
  24. T. Silver, S. Dan, K. Srinivas, J. B. Tenenbaum, L. P. Kaelbling, and M. Katz, “Generalized planning in pddl domains with pretrained large language models,” 2023.
  25. T. Birr, C. Pohl, A. Younes, and T. Asfour, “Autogpt+p: Affordance-based task planning with large language models,” 2024.
  26. T. Silver, V. Hariprasad, R. S. Shuttleworth, N. Kumar, T. Lozano-Pérez, and L. P. Kaelbling, “PDDL planning with pretrained large language models,” in NeurIPS Workshop on Foundation Models for Decision Making, 2022.
  27. L. Guan, K. Valmeekam, S. Sreedharan, and S. Kambhampati, “Leveraging pre-trained large language models to construct and utilize world models for model-based task planning,” in Advances in Neural Information Processing Systems, A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., vol. 36.   Curran Associates, Inc., 2023, pp. 79 081–79 094.
  28. Z. Zhao, W. S. Lee, and D. Hsu, “Large language models as commonsense knowledge for large-scale task planning,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  29. S. Izquierdo-Badiola, G. Canal, C. Rizzo, and G. Alenyà, “Improved task planning through failure anticipation in human-robot collaboration,” in 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 7875–7880.
  30. S. Yang, X. Mao, Q. Wang, and Y. Huang, “A hybrid planning approach for accompanying information-gathering in plan execution monitoring,” Journal of Intelligent & Robotic Systems, vol. 103, 2021.
  31. E. Rohmer, S. P. N. Singh, and M. Freese, “V-rep: A versatile and scalable robot simulation framework,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 1321–1326.
  32. T. Brown, B. Mann, N. Ryder, M. Subbiah, and et al, “Language models are few-shot learners,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33.   Curran Associates, Inc., 2020, pp. 1877–1901.
  33. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” 2023.
  34. S. Richter and M. Westphal, “The lama planner: guiding cost-based anytime planning with landmarks,” J. Artif. Int. Res., vol. 39, no. 1, pp. 127––177, sep 2010.
  35. M. Helmert, “The fast downward planning system,” Journal of Artificial Intelligence Research, vol. 26, pp. 191––246, Jul. 2006.
  36. I. A. Şucan, M. Moll, and L. E. Kavraki, “The Open Motion Planning Library,” IEEE Robotics & Automation Magazine, vol. 19, no. 4, pp. 72–82, December 2012.
  37. G. Team, R. Anil, S. Borgeaud, and et al., “Gemini: A family of highly capable multimodal models,” 2023.
  38. Anthropic, “Claude 3: A family of large, capable, and steerable models,” https://www.anthropic.com/news/claude-3-family, Mar. 2023, accessed: 16 March, 2024.
  39. OpenAI, “Gpt-4 technical report,” ArXiv, vol. abs/2303.08774, 2023.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free