Towards LLM Agents for Model-Based Engineering: A Case in Transformation Selection

International audience ; In Model-Based Engineering (MBE), practitioners frequently face the challenge of selecting appropriate tools from a large number of options. This requires both deep domain-specific knowledge and technical expertise. LLM-based agents are software components that depend on Large Language Models (LLMs) to autonomously select and apply software tools to perform specific tasks. Although LLMs have already been applied to support various MBE activities, considering LLM-based agents to autonomously assist users of MBE tools remains underexplored. This is particularly challenging in industrial MBE environments where only medium-sized on-premise LLMs can be used due to company policies related to security or data privacy (for instance). To investigate the potential of LLM-based agents for MBE, we start with model-to-model transformation as a core MBE technique. Currently, off-the-shelf agents such as Microsoft Copilot can invoke a transformation engine (e.g., ATL) when the task is explicitly described. However, these agents struggle to select the correct transformation when they only have limited contextual information, especially when coupled with medium-size LLMs. To overcome this, we propose an approach based on complementary mechanisms. First, we build a model transformation server and an LLM agent with dedicated tools for each transformation available on the server. Second, to enable the agent to efficiently select transformations, we rely on a tool retrieval technique based on a tool relevance score computed by an LLM. We evaluate our LLM agent on a transformation dataset we also contribute to the community.Our comparative study shows that the newly proposed LLM agent responds more accurately to user instructions.