DocETL – AI-Powered Document ETL Platform

# DocETL System Description and LLM Instructions (Short) Note: use docetl.org/llms-full.txt for the full system description and LLM instructions. DocETL is a system for creating and executing LLM-powered data processing pipelines, designed for complex document processing tasks. It provides a low-code, declarative YAML interface to define complex data operations on unstructured datasets. DocETL is built and maintained by the EPIC lab at UC Berkeley. Learn more at https://www.docetl.org. We have an integrated development environment for building and testing pipelines, at https://www.docetl.org/playground. Our IDE is called DocWrangler. ## Docs - [LLM Instructions (Full)](https://www.docetl.org/llms-full.txt) - [Website](https://www.docetl.org) - [DocWrangler Playground](https://www.docetl.org/playground) - [Main Documentation](https://ucbepic.github.io/docetl) - [GitHub Repository](https://github.com/ucbepic/docetl) - [Agentic Optimization Research Paper](https://arxiv.org/abs/2410.12189) - [Discord Community](https://discord.gg/fHp7B2X3xx) ### Core Operators - [Map Operation](https://ucbepic.github.io/docetl/operators/map/) - [Reduce Operation](https://ucbepic.github.io/docetl/operators/reduce/) - [Resolve Operation](https://ucbepic.github.io/docetl/operators/resolve/) - [Parallel Map Operation](https://ucbepic.github.io/docetl/operators/parallel-map/) - [Filter Operation](https://ucbepic.github.io/docetl/operators/filter/) - [Equijoin Operation](https://ucbepic.github.io/docetl/operators/equijoin/) ### Auxiliary Operators - [Split Operation](https://ucbepic.github.io/docetl/operators/split/) - [Gather Operation](https://ucbepic.github.io/docetl/operators/gather/) - [Unnest Operation](https://ucbepic.github.io/docetl/operators/unnest/) - [Sample Operation](https://ucbepic.github.io/docetl/operators/sample) - [Code Operation](https://ucbepic.github.io/docetl/operators/code/) ### LLM Providers - [LiteLLM Supported Providers](https://docs.litellm.ai/docs/provid