Drone Analytics for Everyone: Chatbots for Easy Access Analytics and AI using Drones (MSc)

Short description

Drone analytics has the potential to contribute significantly to sustainable development across various sectors. From eye-in-the-sky object identification to pest control and forest inventories, the unmanned aerial vehicle (UAV) is becoming a highly flexible tool for many environmental applications. However, the flexibility comes at a cost, before the insights and applications are developed, many decisions need to be made on what equipment to get, which sensors to use, what models to apply, and which outputs to work towards. The transparency on what to get, what to expect, and how to get started is seen as a barrier to entry for drone analytics.

According to previous research, large language models (LLMs), such as ChatGPT, offer contextual, user-friendly and nuanced interpretations in complex optimization scenarios. Especially retrieval augmented generation (RAG) offers the ability to connect background information into the LLM using the langchain Python library.

In this research assignment, we aim to develop a decision support system using LLMs in combination with expert-information on drone analytics, aiming to lower the barrier to entry. The assignment is embedded in a large EU project called ICAERUS. The ICAERUS vision is to explore drone-based opportunities and provide a more complete and interconnected account of their potential and impacts as multi-purpose vehicles in EU agriculture, forestry, and rural areas.

Objectives

1. Review previous work on decision support systems and/or LLMs, particularly for precision agriculture and drone analytics,
2. Explore the ways to use and fine-tune LLMs on drone analytics data,
3. Design and implement a decision support system through chatbots to help users choose a suitable drone analytics scenario and configuration for their specific use case and goals,
4. Evaluate the effectiveness and usability of the solution in the context of the ICAERUS project.

Tasks

The work in this master thesis entails:

Literature Review:

• Conduct an in-depth review of prior work on decision support systems with LLMs
• Specify the role of large language models (LLMs) in precision agriculture and drone analytics.

LLM Exploration:

• Explore tuning approaches for LLMs to work with metadata and documentation of existing backends
• Implement methods for fine-tuning LLMs on metadata from drone-specific datasets and analytical models.

System Design and Implementation:

• Conceptualize a decision support system integrating chatbot technology.
• Develop the chatbot to guide users in selecting suitable drone analytics scenarios and configurations tailored to their specific use cases and goals.

Evaluation:

• Assess the effectiveness of the developed decision support system.
• Evaluate its usability within the context of the ICAERUS project

Literature

• Haurum, K. R., Ma, R., & Long, W. (2024). Real Estate with AI: An agent based on LangChain. Procedia Computer Science, 242, 1082-1088.
• Scotti, V., & Carman, M. J. (2024, August). LLM Support for Real-Time Technical Assistance. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 388-393). Cham: Springer Nature Switzerland.
• Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., ... & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33, 9459-9474.

Requirements

• Courses: Programming in Python (INF-22306), with more programming knowledge acquired in Big Data (INF-34306), Data Science Concepts (INF-34306), Machine Learning (FTE-35306) or similar courses
• Required skills/knowledge: basic data analytics, interest in decision support systems, analytical skills

Key words: LLMs, chatbot, Data Analysis, Data Analytics, Drones, Artificial Intelligence, Decision Support Systems

Contact person(s)

Jurrian Doornbos (jurriandoornbos@wur.nl)

Önder Babur (onder.babur@wur.nl)