apache_beam.ml.rag package

Apache Beam RAG (Retrieval Augmented Generation) components. This package provides components for building RAG pipelines in Apache Beam, including: - Chunking - Embedding generation - Vector storage - Vector search enrichment

Subpackages

• apache_beam.ml.rag.chunking package
  • Submodules
    • apache_beam.ml.rag.chunking.base module
      • ChunkingTransformProvider
    • apache_beam.ml.rag.chunking.langchain module
      • LangChainChunker
• apache_beam.ml.rag.embeddings package
  • Submodules
    • apache_beam.ml.rag.embeddings.base module
      • create_rag_adapter()
    • apache_beam.ml.rag.embeddings.huggingface module
      • HuggingfaceTextEmbeddings
    • apache_beam.ml.rag.embeddings.vertex_ai module
      • VertexAITextEmbeddings
• apache_beam.ml.rag.enrichment package
  • Submodules
    • apache_beam.ml.rag.enrichment.bigquery_vector_search module
      • BigQueryVectorSearchParameters
      • BigQueryVectorSearchEnrichmentHandler
      • join_fn()
• apache_beam.ml.rag.ingestion package
  • Submodules
    • apache_beam.ml.rag.ingestion.alloydb module
      • AlloyDBConnectionConfig
      • ConflictResolution
      • chunk_embedding_fn()
      • chunk_content_fn()
      • chunk_metadata_fn()
      • ColumnSpec
      • chunk_id_fn()
      • ColumnSpecsBuilder
      • AlloyDBVectorWriterConfig
    • apache_beam.ml.rag.ingestion.base module
      • VectorDatabaseWriteConfig
      • VectorDatabaseWriteTransform
    • apache_beam.ml.rag.ingestion.bigquery module
      • SchemaConfig
      • BigQueryVectorWriterConfig

Submodules

• apache_beam.ml.rag.types module
  • Content
    • Content.text
  • Embedding
    • Embedding.dense_embedding
    • Embedding.sparse_embedding
  • Chunk
    • Chunk.content
    • Chunk.id
    • Chunk.index
    • Chunk.metadata
    • Chunk.embedding