Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard
Source Entity
Hacker News
A multi-agent system called Memdoor, utilizing five Qwen3.8-27B models, has outperformed the Fable 5 model on LiveCodeBench Hard. This automated programming workflow demonstrates the efficacy of collaborative, disk-based multi-agent architectures.
The Rise of Multi-Agent Programming Frameworks
The emergence of Memdoor marks a significant shift in how Large Language Models (LLMs) are applied to complex software engineering tasks. By employing a multi-agent harness that utilizes five Qwen3.8-27B models, the system effectively replicates and improves upon the GVS5H methodology. Unlike monolithic models that attempt to solve problems in a single pass, Memdoor distributes cognitive labor among specialized roles, including a planner, two ideators, a solver, and a manager, alongside a dedicated repair worker.
The Architecture of Collaboration
What distinguishes Memdoor from traditional programming assistants is its rigid, file-based communication protocol. Each agent operates from a fresh context and interacts with its counterparts exclusively by reading and writing to the disk. This approach mimics human developer workflows where state is managed through persistent files rather than ephemeral memory. By stripping away direct model-to-model chatter and relying on a shell script to validate success, the system ensures that performance is measured by actual code execution rather than subjective model confidence.
Benchmarking Against Industry Standards
In recent testing on the LiveCodeBench Hard dataset, this five-agent configuration demonstrated remarkable robustness. Replicating the authors' exact prompts on the Qwen3.8-27B architecture, the workflow achieved a mean score of 9.8 out of 10 across five passes on the first ten problems. This performance notably outpaced the 8.0 score achieved by Fable 5 alone and even edged out the original harness performance of 9.6, suggesting that the integration of this specific multi-agent structure optimizes the underlying model's potential.
The Autonomy Factor
A particularly compelling aspect of this development is the degree of automation involved. The code powering the Memdoor workflow was generated by the models themselves based on a high-level description of the method, rather than being hand-coded by human engineers. This 'self-bootstrapping' capability highlights a future where AI systems can design their own collaborative frameworks to tackle increasingly difficult technical challenges, effectively removing the human bottleneck in system architecture.
Broader Implications and Future Trends
The success of the Memdoor workflow signals a broader trend toward modular, agentic systems in artificial intelligence. As models like Qwen3.8-27B become more accessible, the focus of AI research is shifting from simply increasing parameter counts to optimizing how multiple specialized instances communicate and verify one another's work. This methodology suggests that even medium-sized models can outperform larger competitors when organized into a disciplined, iterative, and validation-focused harness.
Conclusion
In summary, the Memdoor project provides a blueprint for leveraging collective intelligence in machine learning. By prioritizing empirical success—verified via shell scripts—over model-generated justifications, the project sets a high bar for future programming benchmarks. As these multi-agent frameworks continue to evolve, we can expect to see a transition toward autonomous coding environments capable of sustained, high-level problem solving without constant human intervention.