> ## Documentation Index
> Fetch the complete documentation index at: https://docs.soarlabs.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# Upload a File

> Attach one or more files to a corpus so they can be processed and indexed.

## Overview

This endpoint handles binary file uploads and triggers asynchronous ingestion (content extraction + chunking + vector indexing). Only corpora you own can accept uploads, and files must be in supported formats.

<Info>
  **Supported Formats**: PDF, DOCX, TXT, CSV, JSON, Markdown (MD), Excel (XLSX/XLS), HTML/HTM, LOG files
</Info>

<Warning>
  Files are processed asynchronously. Monitor `indexing_status` to track when files are ready for querying.
</Warning>

## Authentication

Requires valid JWT token or session authentication. You must be the owner of the target corpus.

## Request Body

<ParamField body="corpora" type="UUID" required>
  ID of the corpus that will own these files. Must be a corpus you created and have access to.
</ParamField>

<ParamField body="files" type="file[]" required>
  One or more files to upload. Send as `multipart/form-data` with multiple `files` fields.

  **Processing pipeline:**

  1. File validation (format, size)
  2. Upload to cloud storage
  3. Content extraction (text, tables, images)
  4. Chunking and metadata generation
  5. Vector embedding and indexing

  **File size limits**: Check your instance configuration (typically 50MB per file)
</ParamField>

## Example request

```bash theme={null}
curl -X POST https://{your-host}/api/data/files/ \
  -H "Authorization: Bearer $SOAR_LABS_TOKEN" \
  -F "corpora=8d0f0a5d-4b5e-4c09-9db6-0e9d2aa8a9fd" \
  -F "files=@playbook.pdf" \
  -F "files=@metrics.csv"
```

## Response

Returns an array of file objects (one for each uploaded file):

<ResponseField name="id" type="UUID">
  Unique identifier for the file resource. Use for tracking, retrieval, or deletion.
</ResponseField>

<ResponseField name="created_at" type="timestamp">
  ISO 8601 timestamp when the file was uploaded.
</ResponseField>

<ResponseField name="updated_at" type="timestamp">
  Last update timestamp. Changes when indexing status updates.
</ResponseField>

<ResponseField name="indexed_on" type="timestamp | null">
  Timestamp when indexing completed successfully. `null` while processing.
</ResponseField>

<ResponseField name="indexing_status" type="string">
  Current processing status of the file:

  * `PRS` - Processing (extraction and indexing in progress)
  * `IND` - Indexed (ready for queries)
  * `ERR` - Error (processing failed, check logs)
  * `PND` - Pending (queued for processing)
</ResponseField>

<ResponseField name="file_name" type="string">
  Original filename as uploaded.
</ResponseField>

<ResponseField name="file_type" type="string">
  Detected file extension/type (e.g., `pdf`, `docx`, `csv`).
</ResponseField>

<ResponseField name="location" type="string (URL)">
  Cloud storage URL where the file is persisted. Access requires authentication.
</ResponseField>

<ResponseField name="file_size" type="float">
  File size in bytes.
</ResponseField>

<ResponseField name="corpora" type="UUID">
  ID of the parent corpus containing this file.
</ResponseField>

## Example Response

```json theme={null}
[
  {
    "id": "0f75c73e-91bb-4e2b-9ff2-6820a8636ad8",
    "created_at": "2024-09-01T11:23:12.884Z",
    "updated_at": "2024-09-01T11:23:12.884Z",
    "indexed_on": null,
    "indexing_status": "PRS",
    "file_name": "playbook.pdf",
    "file_type": "pdf",
    "location": "https://storage/.../playbook.pdf",
    "file_size": 180423.0,
    "corpora": "8d0f0a5d-4b5e-4c09-9db6-0e9d2aa8a9fd"
  },
  {
    "id": "f5b69026-879f-457d-8b5a-78e20a8c912c",
    "created_at": "2024-09-01T11:23:12.901Z",
    "updated_at": "2024-09-01T11:23:12.901Z",
    "indexed_on": null,
    "indexing_status": "PRS",
    "file_name": "metrics.csv",
    "file_type": "csv",
    "location": "https://storage/.../metrics.csv",
    "file_size": 2048.0,
    "corpora": "8d0f0a5d-4b5e-4c09-9db6-0e9d2aa8a9fd"
  }
]
```

## Client examples

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import os
    import requests

    BASE_URL = "https://your-soar-instance.com"
    TOKEN = os.environ["SOAR_LABS_TOKEN"]
    CORPUS_ID = "8d0f0a5d-4b5e-4c09-9db6-0e9d2aa8a9fd"

    files = [
        ("files", ("playbook.pdf", open("playbook.pdf", "rb"), "application/pdf")),
        ("files", ("metrics.csv", open("metrics.csv", "rb"), "text/csv")),
    ]

    response = requests.post(
        f"{BASE_URL}/api/data/files/",
        headers={"Authorization": f"Bearer {TOKEN}"},
        data={"corpora": CORPUS_ID},
        files=files,
        timeout=120,
    )
    response.raise_for_status()
    uploaded_files = response.json()
    ```
  </Tab>

  <Tab title="TypeScript / JavaScript">
    ```ts theme={null}
    const BASE_URL = "https://your-soar-instance.com";
    const token = process.env.SOAR_LABS_TOKEN!;

    async function uploadFiles(corpusId: string, blobs: File[]) {
      const form = new FormData();
      form.append("corpora", corpusId);
      blobs.forEach((blob) => form.append("files", blob));

      const response = await fetch(`${BASE_URL}/api/data/files/`, {
        method: "POST",
        headers: {
          Authorization: `Bearer ${token}`,
        },
        body: form,
      });

      if (!response.ok) {
        throw new Error(`Upload failed: ${response.status}`);
      }

      return response.json();
    }
    ```
  </Tab>

  <Tab title="Java">
    ```java theme={null}
    import java.io.IOException;
    import java.net.URI;
    import java.net.http.HttpClient;
    import java.net.http.HttpRequest;
    import java.net.http.HttpResponse;
    import java.nio.file.Path;
    import java.util.List;

    // Use a multipart helper (e.g., Apache HttpClient or OkHttp) for production.
    // Simplified example with java.net.http sending a single file.
    var BASE_URL = "https://your-soar-instance.com";
    var token = System.getenv("SOAR_LABS_TOKEN");
    var corpusId = "8d0f0a5d-4b5e-4c09-9db6-0e9d2aa8a9fd";

    String boundary = "----SoarBoundary";
    var body = "--" + boundary + "\r\n" +
        "Content-Disposition: form-data; name=\"corpora\"\r\n\r\n" + corpusId + "\r\n" +
        "--" + boundary + "\r\n" +
        "Content-Disposition: form-data; name=\"files\"; filename=\"playbook.pdf\"\r\n" +
        "Content-Type: application/pdf\r\n\r\n" +
        java.nio.file.Files.readString(Path.of("playbook.pdf")) +
        "\r\n--" + boundary + "--\r\n";

    var request = HttpRequest.newBuilder(URI.create(BASE_URL + "/api/data/files/"))
        .header("Authorization", "Bearer " + token)
        .header("Content-Type", "multipart/form-data; boundary=" + boundary)
        .POST(HttpRequest.BodyPublishers.ofString(body))
        .build();

    var response = HttpClient.newHttpClient().send(request, HttpResponse.BodyHandlers.ofString());

    if (response.statusCode() >= 400) {
        throw new RuntimeException("Upload failed: " + response.statusCode());
    }
    ```
  </Tab>
</Tabs>

## Post-Upload Operations

<AccordionGroup>
  <Accordion title="Monitor Indexing Status" icon="pulse">
    Poll the files endpoint to track processing progress:

    ```bash theme={null}
    curl -X GET "https://{your-host}/api/data/files/?corpora={corpus-id}" \
      -H "Authorization: Bearer $SOAR_LABS_TOKEN"
    ```

    **Status progression:**
    `PND` → `PRS` → `IND` (success) or `ERR` (failure)

    **Typical processing times:**

    * Small text files (\< 1MB): 5-15 seconds
    * PDFs with images (5-10MB): 30-60 seconds
    * Large documents (20MB+): 2-5 minutes
  </Accordion>

  <Accordion title="Retrieve File Details" icon="file-lines">
    Get detailed information about a specific file:

    ```bash theme={null}
    curl -X GET "https://{your-host}/api/data/files/{file-id}/" \
      -H "Authorization: Bearer $SOAR_LABS_TOKEN"
    ```

    Returns full metadata including chunking statistics and extraction details.
  </Accordion>

  <Accordion title="Delete Files" icon="trash">
    Remove files from the corpus and vector index:

    ```bash theme={null}
    curl -X DELETE "https://{your-host}/api/data/files/{file-id}/" \
      -H "Authorization: Bearer $SOAR_LABS_TOKEN"
    ```

    **Important**: Deletion is immediate and irreversible. The operation:

    * Removes the file from cloud storage
    * Deletes all associated vector embeddings
    * Updates corpus size metadata
    * Cannot be undone - you'll need to re-upload if deleted accidentally
  </Accordion>

  <Accordion title="Handle Processing Errors" icon="triangle-exclamation">
    If a file shows `indexing_status: "ERR"`, common causes include:

    * **Corrupted or invalid file format** - Re-export and try again
    * **Unsupported encoding** - Convert to UTF-8 for text files
    * **Password-protected PDFs** - Remove protection before uploading
    * **Extremely large files** - Split into smaller chunks
    * **Unsupported content** - Check if file type is in supported list

    To retry: Delete the failed file and re-upload with corrections.
  </Accordion>
</AccordionGroup>

## Best Practices

<Tip>
  **Batch Uploads**: Upload multiple related files in a single request to improve efficiency and reduce API calls.
</Tip>

<AccordionGroup>
  <Accordion title="Optimize File Preparation" icon="wand-magic-sparkles">
    Before uploading:

    1. **Remove unnecessary pages** - Reduce file size by excluding cover pages, blank pages
    2. **Use OCR for scanned PDFs** - Convert image-based PDFs to searchable text
    3. **Clean up formatting** - Remove excessive whitespace, fix broken tables
    4. **Verify encoding** - Ensure text files use UTF-8 encoding
    5. **Test file opens** - Verify files aren't corrupted before upload
  </Accordion>

  <Accordion title="Organize by Corpus" icon="folder-tree">
    Create separate corpora for different content types or use cases:

    * **Internal Documentation** - Company policies, procedures
    * **Product Knowledge** - Technical specs, user guides
    * **Customer Support** - FAQs, troubleshooting guides
    * **Training Materials** - Onboarding docs, tutorials

    This improves query accuracy and makes management easier.
  </Accordion>

  <Accordion title="Monitor Upload Queue" icon="list-check">
    For bulk uploads:

    1. Upload in reasonable batches (10-20 files per request)
    2. Poll status every 10-30 seconds
    3. Implement exponential backoff if servers are busy
    4. Log file IDs for tracking and error recovery
  </Accordion>
</AccordionGroup>

<Warning>
  **Rate Limits**: Large file uploads may hit rate limits. Implement exponential backoff and retry logic in production systems.
</Warning>


## OpenAPI

````yaml POST /api/data/files/
openapi: 3.0.3
info:
  title: Soar Labs API - Insider Dev preview
  version: 0.1.0
  description: <b>Soar Labs Advanced RAG platform</b>
servers: []
security: []
paths:
  /api/data/files/:
    post:
      tags:
        - Resources
      description: >-
        File creation API using Django Rest Framework.


        This viewset provides CRUD operations for File objects associated with a
        Corpora.
      operationId: api_data_files_create
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/File'
          application/x-www-form-urlencoded:
            schema:
              $ref: '#/components/schemas/File'
          multipart/form-data:
            schema:
              $ref: '#/components/schemas/File'
        required: true
      responses:
        '201':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/File'
          description: ''
      security:
        - jwtHeaderAuth: []
        - jwtCookieAuth: []
        - cookieAuth: []
        - basicAuth: []
components:
  schemas:
    File:
      type: object
      properties:
        id:
          type: string
          format: uuid
          readOnly: true
        created_at:
          type: string
          format: date-time
          readOnly: true
          description: The date and time the organization was created
        updated_at:
          type: string
          format: date-time
          readOnly: true
          description: Last updated time
        indexed_on:
          type: string
          format: date-time
          readOnly: true
          nullable: true
        indexing_status:
          allOf:
            - $ref: '#/components/schemas/IndexingStatusEnum'
          readOnly: true
        file_name:
          type: string
          readOnly: true
          title: Original File Name
          description: Original, user-facing name of the uploaded file.
        file_type:
          type: string
          readOnly: true
          nullable: true
          description: Type of the file
        location:
          type: string
          format: uri
          title: File Location
          description: Location of the file on Remote Storage
        file_size:
          type: number
          format: double
          readOnly: true
          nullable: true
          description: bytes
        corpora:
          type: string
          format: uuid
          description: Corpora to which the Maps to
      required:
        - corpora
        - created_at
        - file_name
        - file_size
        - file_type
        - id
        - indexed_on
        - indexing_status
        - location
        - updated_at
    IndexingStatusEnum:
      enum:
        - PND
        - IQE
        - PRS
        - DEX
        - DER
        - IND
        - CMP
        - ERR
      type: string
      description: |-
        * `PND` - Pending
        * `IQE` - In Queue
        * `PRS` - Processing
        * `DEX` - Data Extracted Successfully
        * `DER` - Data Extraction Error
        * `IND` - Indexed
        * `CMP` - Completed
        * `ERR` - Error
  securitySchemes:
    jwtHeaderAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT
    jwtCookieAuth:
      type: apiKey
      in: cookie
      name: soar-app-auth
    cookieAuth:
      type: apiKey
      in: cookie
      name: sessionid
    basicAuth:
      type: http
      scheme: basic

````