Class DocumentCompressorPipeline

Document compressor that uses a pipeline of Transformers.

Example

const compressorPipeline = new DocumentCompressorPipeline({
  transformers: [
    new RecursiveCharacterTextSplitter({
      chunkSize: 200,
      chunkOverlap: 0,
    }),
    new EmbeddingsFilter({
      embeddings: new OpenAIEmbeddings(),
      similarityThreshold: 0.8,
      k: 5,
    }),
  ],
});
const retriever = new ContextualCompressionRetriever({
  baseCompressor: compressorPipeline,
  baseRetriever: new TavilySearchAPIRetriever({
    includeRawContent: true,
  }),
});
const retrievedDocs = await retriever.getRelevantDocuments(
  "What did the speaker say about Justice Breyer in the 2022 State of the Union?",
);
console.log({ retrievedDocs });

Hierarchy

Constructors

constructor

Properties

transformers

Methods

compressDocuments isBaseDocumentCompressor

Constructors

constructor

Properties

transformers

transformers: (BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]> | BaseDocumentCompressor)[]

Methods

compressDocuments

  • Abstract method that must be implemented by any class that extends BaseDocumentCompressor. This method takes an array of Document objects and a query string as parameters and returns a Promise that resolves with an array of compressed Document objects.

    Parameters

    • documents: DocumentInterface<Record<string, any>>[]

      An array of Document objects to be compressed.

    • query: string

      A query string.

    • Optional callbacks: Callbacks

    Returns Promise<DocumentInterface<Record<string, any>>[]>

    A Promise that resolves with an array of compressed Document objects.

Static isBaseDocumentCompressor

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc