DocETL – AI-Powered Document ETL Platform

# DocETL System Description and LLM Instructions (Short) Note: use https://www.docetl.org/llms-full.txt for the full system description and LLM instructions. Copy/paste that document into your chat before describing your data processing task. DocETL helps you process large collections of data (structured and unstructured) with LLMs. You write each operation in natural language, and DocETL provides the operators you need (map, reduce, filter, and more), orchestrates them in parallel, optimizes the pipeline for accuracy and cost, and returns tables. Pipelines can be written in YAML (low-code) or Python (the Frame API). Both are first class. DocETL is built and maintained by the EPIC Data Lab at UC Berkeley. Learn more at https://www.docetl.org. ## Docs - [LLM Instructions (Full)](https://www.docetl.org/llms-full.txt) - [Main Documentation](https://ucbepic.github.io/docetl) - [Python API Reference](https://ucbepic.github.io/docetl/api-reference/python/) - [GitHub Repository](https://github.com/ucbepic/docetl) - [DocWrangler Playground](https://www.docetl.org/playground) - [Discord Community](https://discord.gg/fHp7B2X3xx) ### Core Concepts - [Pipelines](https://ucbepic.github.io/docetl/concepts/pipelines/) - [Datasets & Frames](https://ucbepic.github.io/docetl/concepts/datasets/) - [Operators & Validation](https://ucbepic.github.io/docetl/concepts/operators/) - [Output Schemas](https://ucbepic.github.io/docetl/concepts/schemas/) ### Operators - [Map](https://ucbepic.github.io/docetl/operators/map/), [Reduce](https://ucbepic.github.io/docetl/operators/reduce/), [Filter](https://ucbepic.github.io/docetl/operators/filter/), [Resolve](https://ucbepic.github.io/docetl/operators/resolve/), [Equijoin](https://ucbepic.github.io/docetl/operators/equijoin/), [Extract](https://ucbepic.github.io/docetl/operators/extract/), [Parallel Map](https://ucbepic.github.io/docetl/operators/parallel-map/) - [Split](https://ucbepic.github.io/docetl/operators/split/), [Gather](https:/