Problems when working with large documents

[jesusSerranoCarballido]

Hello everyone, and thanks in advance.

I'm learning how kernel memory works and I'm having trouble. I've been testing, and I don't have a problem with small documents (up to 3 sheets, I just import them with ImportDocumentAsync). The problem is when the documents are large. I see that the tables don't provide the information, and I can't create the corresponding chunks. I've tried WithCustomTextPartitioningOptions, but it doesn't work. In the end, I did
"var chunks = TextChunker.SplitMarkDownLines(text, maxTokensPerLine: 500);
for (var i = 0; i < chunks.Count; i++) {
documentId = expectMemory.ImportTextAsync(chunks[i]);
}"
But I don't think it's correct, and I'm not approaching it correctly.

I've already mentioned that this project is for learning, and I'm doing everything in memory or on disk using Olama.

I'm posting the code, just in case it's necessary.

Program.cs
var t =builder.Services.AddKernelMemory(x => {
    x
    .WithOllamaTextGeneration(OllamaConfig())
    .WithOllamaTextEmbeddingGeneration(OllamaConfig())
    /*.WithCustomTextPartitioningOptions(new TextPartitioningOptions() {
        MaxTokensPerParagraph = 100,
        OverlappingTokens = 0
    })  */
    .WithSearchClientConfig(new SearchClientConfig() {
        EmptyAnswer = "Lo siento, no he encontrado ninguna información relevante que pueda utilizarse para responder a su pregunta.",
    })
    .WithSimpleFileStorage(new SimpleFileStorageConfig() {
        StorageType = Microsoft.KernelMemory.FileSystem.DevTools.FileSystemTypes.Volatile,
    })
    .WithSimpleVectorDb(new SimpleVectorDbConfig() {
        StorageType=Microsoft.KernelMemory.FileSystem.DevTools.FileSystemTypes.Volatile,
    })
    .WithSimpleQueuesPipeline(new Microsoft.KernelMemory.Pipeline.Queue.DevTools.SimpleQueuesConfig() {
        StorageType=Microsoft.KernelMemory.FileSystem.DevTools.FileSystemTypes.Volatile,

    });
}); 
private static OllamaConfig OllamaConfig() {
    return new OllamaConfig {
        Endpoint = "http://localhost:11434",
        TextModel = new OllamaModelConfig("mistral:7b-instruct-v0.3-q4_1", 131072),
        EmbeddingModel = new OllamaModelConfig("mistral:7b-instruct-v0.3-q4_1", 2048)
    };
}

post request
await memory.ImportDocumentAsync("C:\\temp\\02-lite.pdf");
var answer = await memory.AskAsync(question.Text);  => good is lite document


var chunks = TextChunker.SplitMarkDownLines(text, maxTokensPerLine: 500);
for (var i = 0; i < chunks.Count; i++) {
    documentId = await memory.ImportTextAsync(chunks[i]);
}
var answer = await memory.AskAsync(question.Text);
``



Thanks for all the help possible

[jesusSerranoCarballido]

problems with context is not adecuado