JSON Truncation in Spring AI Streaming Response with Large Sets

[LiveNathan]

Bug description

When using Spring AI's streaming client with larger sets of keywords (~1000), the response is truncated mid-JSON object, resulting in a JsonMappingException. The same code works perfectly with smaller sets (~100 keywords). There's no clear indication if this is due to token limits or other constraints, making it difficult to handle appropriately in production code.

Environment

• Spring AI version: 1.0.0-M5
• Spring Boot version: 3.4.2
• Java version: 23
• Model: GPT_4_O_MINI

Steps to reproduce

1. Create a test with ~100 keywords (passes) and another with ~1000 keywords (fails)
2. Use Spring AI's streaming client (required to avoid timeout issues as per discussion ReadTimeoutException with Spring AI for Moderate-Sized Prompts (<4K characters) #2195)
3. Attempt to process the response

Expected behavior

Either:

1. Successfully process all keywords, or
2. Receive a clear error indicating why the processing failed (e.g., token limit exceeded)

Minimal Complete Reproducible example

Here are two tests - one that passes with ~100 keywords and one that fails with larger sets:

Test that passes with ~100 keywords

@Test
    void givenLongResponseTime_whenProcessingWithStreamingChatClient_thenResponseIsSuccessful() {
        ChatClient chatClient = ChatClient.create(chatModel);
        var converter = new BeanOutputConverter<>(new ParameterizedTypeReference<AiResponseKeywordsAligned>() {
        });
        String systemPrompt = "You are a keyword research assistant specialized in SEO and Google Keyword Planner, helping users discover high-quality, long-tail keywords for niche businesses. {format}";
        String userMessage = """
                ## How to rate industry alignment
                Using the following scale, rate how closely each keyword aligns with the industry. The specific values represent key reference points or milestones, but your rating should lie on the continuous scale between 0 and 100 in proximity to those milestones.
                100 = Perfect alignment: The keyword is highly relevant to the food and beverage, cafe culture, coffee shops industry.
                50  = Partial alignment: The keyword is somewhat related or adjacent to this industry.
                0   = Misalignment: The keyword is not related to this industry at all. In addition, ensure that keywords related to  are give a low rating.
                
                ## List of keywords to analyze
                150 ml to ounces, 150ml to ounces, 20 oz in pounds, 5 ounces to g, 5 ounces to gr, 5 ounces to grams, 6'2 in cm, 80 ml to ounces, a gallon of water is how many water bottles, bahama mama recipe, benefits of central business district, best afternoon tea london, best cafe near me, best espresso martini recipe, best local coffee shops, blue in spanish, cafe in meiner nähe, cafes to study, caffeine in diet coke, caffeine in espresso shot, chai tea benefits, chocolate cafe, chocolate house, chocolate rain, chocolate rain song, chocolate world, cinnamon tea benefits, coffee loophole recipe, coffee shops with wifi near me, cranberry in spanish, daiquiri recipe, dark chocolate benefits, definition for wholesome, est to ist, exit to eden, factory tea bar, french martini recipe, fruits start with i, gentleman in moscow book, gin and tonic recipe, ginger tea benefits, green bean casserole with fresh green beans, health food store, hershey chocolate world, hershey's chocolate world, hibiscus tea benefits, homemade hot chocolate, horchata recipe, hot chocolate near me, hot chocolate recipe, hot chocolate run, how many bottle of waters is a gallon, how many bottle waters are in a gallon, how many bottled waters are in a gallon, how many grams of sugar in a tablespoon of sugar, how many grams of sugar in a teaspoon, how many grams of sugar in a teaspoon of sugar, how many ounces of water to drink a day, how many water bottle are in a gallon, how many water bottles are in a gallon, how many water bottles are in a gallon of water, how many water bottles is a gallon of water, how many water bottles is in a gallon, how much caffeine in matcha, how much sugar can of coke, how much sugar i n can of coke, how much sugar in a can of coca cola, how much sugar in a can of coke, how much sugar in can of coke, how much sugar in can of soda, how much sugar is in a can of coke, how much sugar is in a can of pop, how much sugar is in a soda can, how much sugar is in coke can, how much sugars in a can of coke, how to descale a keurig, how to make hot chocolate, how to make hummingbird nectar, hugo spritz recipe, ingredients in coke, is carbonated water bad for you, is coconut water good for you, is coffee good for you, is coke zero bad for you, is dark chocolate good for you, is diet coke bad for you, is fluoride bad for you, is matcha good for you, is oat milk good for you, is sparkling water good for you, is vitamin water good for you, juice bar, kakawa chocolate house, lemon drop recipe, loco moco recipe, long island iced tea recipe, lychee martini recipe, lyrics to escape the pina colada song, mai tai recipe, mango smoothie recipe, martini recipe, matcha benefits, mg of caffeine in coffee, movita juice bar, nekter juice bar, oolong tea benefits, oz in grams, peppermint tea benefits, pina colada recipe, pomegranate benefits, raspberry leaf tea benefits, raw milk benefits, tea bar, time for tokyo, time in shan, toothsome chocolate emporium menu, virgin pina colada recipe, water in spanish, what's in spanish, whiskey drink recipes, whole foods coffee bar
                
                ## Output format to return
                Set of keywordsWithThemeAndConfidence objects. Each object must include:
                - The keyword
                - A confidence rating (0–100) indicating how well it aligns with the specified industry.""";
        log.info("User message: {}", userMessage);

        Flux<String> response = chatClient.prompt()
                .system(sp -> sp.text(systemPrompt).param("format", converter.getFormat()))
                .user(userMessage)
                .stream()
                .content();

        assertThat(response)
                .as("Check that the response is not null")
                .isNotNull();

        List<String> chunks = response.collectList().block();
        assertThat(chunks)
                .as("Check that we received multiple chunks")
                .hasSizeGreaterThan(1);

        String content = String.join("", chunks);

        AiResponseKeywordsAligned result = assertDoesNotThrow(() ->
                        converter.convert(content),
                "Response should be convertible to AiResponseKeywordsAligned");
        log.info("Result: {}", result);

        assertThat(result)
                .as("Check that the entity is not null")
                .isNotNull();
        assertThat(result.keywordsWithThemeAndConfidence())
                .as("Check if AI returned any keywords with confidence")
                .isNotNull()
                .isNotEmpty();
    }

Test that fails with ~1000 keywords

@Test
    void given1000Keywords_whenProcessingWithStreamingChatClient_thenResponseIsSuccessful() {
        ChatClient chatClient = ChatClient.create(chatModel);
        var converter = new BeanOutputConverter<>(new ParameterizedTypeReference<AiResponseKeywordsAligned>() {});
        String systemPrompt = "You are a keyword research assistant specialized in SEO and Google Keyword Planner, helping users discover high-quality, long-tail keywords for niche businesses. {format}";

        String keywordsToAnalyze = create1000KeywordsToAnalyze();
        @NotNull Set<String> keywords = Arrays.stream(keywordsToAnalyze.split(","))
                .map(String::trim)
                .collect(Collectors.toSet());
        assertThat(keywords)
                .as("Verify we have exactly 1000 keywords to process")
                .hasSize(1000);

        String userMessage = """
                ## How to rate industry alignment
                Using the following scale, rate how closely each keyword aligns with the industry. The specific values represent key reference points or milestones, but your rating should lie on the continuous scale between 0 and 100 in proximity to those milestones.
                100 = Perfect alignment: The keyword is highly relevant to the food and beverage, cafe culture, coffee shops industry.
                50  = Partial alignment: The keyword is somewhat related or adjacent to this industry.
                0   = Misalignment: The keyword is not related to this industry at all. In addition, ensure that keywords related to  are give a low rating.
                
                ## List of keywords to analyze
                %s
                
                ## Output format to return
                Set of keywordsWithThemeAndConfidence objects. Each object must include:
                - The keyword
                - A confidence rating (0–100) indicating how well it aligns with the specified industry."""
                .formatted(keywordsToAnalyze);
        log.info("User message: {}", userMessage);

        Flux<String> response = chatClient.prompt()
                .system(sp -> sp.text(systemPrompt).param("format", converter.getFormat()))
                .user(userMessage)
                .stream()
                .content();

        assertThat(response)
                .as("Check that the response is not null")
                .isNotNull();

        List<String> chunks = response.collectList().block();
        assertThat(chunks)
                .as("Check that we received multiple chunks")
                .hasSizeGreaterThan(1);

        String content = String.join("", chunks);

        AiResponseKeywordsAligned result = assertDoesNotThrow(() ->
                        converter.convert(content),
                "Response should be convertible to AiResponseKeywordsAligned");
        log.info("Result: {}", result);

        assertThat(result)
                .as("Check that the entity is not null")
                .isNotNull();
        assertThat(result.keywordsWithThemeAndConfidence())
                .as("Check if AI returned any keywords with confidence")
                .isNotNull()
                .isNotEmpty();
    }

Config

@SpringBootConfiguration
public class OpenAiTestConfiguration {

    @Bean
    public OpenAiApi openAiApi() {
        return new OpenAiApi(getApiKey());
    }

    @Bean
    public OpenAiChatModel openAiChatModel(OpenAiApi api) {
        return new OpenAiChatModel(api, OpenAiChatOptions.builder()
                .model(ChatModel.GPT_4_O_MINI)
                .temperature(0.2)
                .build());
    }

    private String getApiKey() {
        String apiKey = System.getenv("SPRING_AI_OPENAI_API_KEY");
        if (!StringUtils.hasText(apiKey)) {
            throw new IllegalArgumentException("You must provide an API key via OPENAI_API_KEY.");
        }
        return apiKey;
    }

}

[LiveNathan]

@NotNull
    private String create1000KeywordsToAnalyze() {
        return "best cafe near me, best cafe wifi study, local coffee shops open now, artisan coffee roasters, specialty coffee beans, cafe breakfast menu, coffee shop pastries, cafe lunch specials, espresso machine reviews, latte art tutorial, cold brew coffee recipe, pour over coffee guide, coffee shop atmosphere, quiet cafes for work, coffee shop happy hour, cafe outdoor seating, bakery cafe combo, coffee shop loyalty program, coffee bean varieties, coffee grinder reviews, cafe barista jobs, coffee shop equipment, coffee shop interior design, cafe music playlist, coffee shop business plan, coffee shop startup costs, cafe food suppliers, coffee shop point of sale, cafe menu design, coffee shop social media, best coffee downtown, coffee shop catering, cafe delivery service, coffee shop marketing, coffee shop decor ideas, cafe franchise opportunities, coffee shop profit margins, cafe staff training, coffee shop gift cards, cafe renovation costs, coffee shop insurance, cafe sanitation guide, coffee shop lease tips, cafe management software, coffee shop suppliers, cafe advertising ideas, coffee shop events, cafe customer service, coffee shop food costs, cafe employee handbook, coffee shop inventory, cafe kitchen equipment, coffee shop accounting, cafe vendor relations, coffee shop regulations, cafe health codes, coffee shop utilities, cafe tax planning, coffee shop licenses, cafe maintenance schedule, coffee shop security, cafe waste management, coffee shop utilities, cafe pest control, coffee shop storage, cafe parking options, coffee shop accessibility, cafe energy efficiency, coffee shop ventilation, cafe water filtration, coffee shop recycling, cafe composting guide, coffee shop sustainability, cafe eco-friendly, coffee shop fair trade, cafe organic options, coffee shop vegan menu, cafe gluten free, coffee shop allergen guide, cafe nutrition facts, coffee shop calorie count, cafe ingredient list, coffee shop dietary info, cafe kosher options, coffee shop halal menu, cafe dairy free, coffee shop sugar free, cafe low carb options, coffee shop keto menu, cafe paleo friendly, coffee shop weight watchers, cafe mediterranean diet, cafe seasonal menu, coffee shop farm to table, cafe local sourcing, coffee shop food waste, cafe portion control, coffee shop recipe costs, cafe inventory tracking, coffee shop order timing, cafe rush hour tips, coffee shop quiet hours, cafe peak times, coffee shop slow periods, cafe customer flow, coffee shop table turnover, cafe seating layout, coffee shop floor plan, cafe kitchen workflow, coffee shop bar setup, cafe cleaning schedule, coffee shop opening tasks, cafe closing duties, coffee shop shift change, cafe cash handling, coffee shop credit cards, cafe mobile payment, coffee shop online orders, cafe pickup window, coffee shop drive through, cafe parking validation, coffee shop tipping guide, cafe minimum wage, coffee shop overtime rules, cafe break policy, coffee shop dress code, coffee shop training manual, cafe safety rules, coffee shop first aid, cafe emergency plan, coffee shop fire safety, cafe security system, coffee shop cameras, cafe alarm system, coffee shop safe handling, cafe money management, french press coffee recipe, cold brew concentrate ratio, nitro cold brew setup, pour over coffee methods, aeropress coffee guide, chemex brewing instructions, siphon coffee tutorial, turkish coffee recipe, vietnamese coffee guide, italian espresso rules, cafe con leche recipe, cuban coffee method, greek coffee preparation, irish coffee recipe, espresso martini mix, affogato dessert recipe, iced latte technique, frozen coffee blends, blended coffee drinks, specialty coffee syrups, coffee flavor combinations, seasonal coffee drinks, pumpkin spice latte recipe, maple coffee blend, caramel coffee drink, mocha sauce recipe, white chocolate mocha, toffee coffee recipe, hazelnut coffee blend, vanilla coffee syrup, cinnamon coffee mix, mint coffee recipe, coconut coffee blend, chocolate coffee drink, coffee smoothie recipe, coffee protein shake, coffee breakfast blend, decaf coffee options, half-caf coffee blend, light roast coffee, medium roast coffee, dark roast coffee, espresso roast beans, coffee origin guide, single origin coffee, coffee blend recipe, arabica coffee beans, robusta coffee guide, coffee cherry tea, coffee bean storage, coffee grind sizes, burr grinder guide, coffee filter types, coffee water ratio, coffee brewing time, coffee temperature guide, coffee cupping guide, coffee tasting notes, coffee flavor wheel, coffee acidity guide, coffee body types, coffee aroma guide, coffee defects list, coffee quality grades, coffee bean density, coffee roast levels, coffee packaging types, coffee vacuum seal, coffee degassing time, coffee shelf life, coffee freshness test, coffee bean moisture, coffee bean density, coffee particle size, coffee extraction rate, coffee brew strength, coffee concentration, coffee total dissolved solids, coffee refractometer, coffee yield calculator, coffee dose weight, coffee shot timing, espresso pressure guide, coffee tamping force, coffee distribution tool, coffee portafilter size, coffee basket types, coffee group head temp, coffee boiler pressure, coffee steam power, milk steaming guide, milk temperature chart, microfoam technique, latte art patterns, coffee drink sizes, coffee cup types, coffee to go cups, coffee sleeve guide, coffee lid options, coffee stirrer types, coffee napkin guide, coffee condiment bar, sugar substitute guide, coffee creamer types, milk alternative guide, oat milk coffee, almond milk latte, soy milk coffee, coconut milk drinks, barista training guide, latte art basics, espresso shot timing, milk steaming tips, coffee bean storage, coffee grinder maintenance, cafe equipment cleaning, coffee machine descaling, coffee shop bookkeeping, cafe social media tips, morning coffee routine, coffee subscription plan, coffee loyalty rewards, best coffee beans online, coffee roasting process, coffee brewing methods, cafe breakfast deals, coffee happy hour times, tea selection guide, herbal tea benefits, green tea caffeine, black tea varieties, chai latte recipe, matcha preparation guide, bubble tea making, kombucha brewing guide, tea storage tips, loose leaf tea guide, tea steeping times, tea temperature chart, cafe sandwich menu, breakfast pastry list, vegan cafe options, gluten free desserts, cafe soup recipes, cafe salad menu, brunch specials cafe, cafe catering menu, coffee cake recipe, muffin varieties cafe, croissant types cafe, bagel sandwich menu, quiche of the day, cafe yogurt parfait, overnight oats cafe, acai bowl recipe, smoothie bowl cafe, protein breakfast cafe, cafe waffle menu, pancake varieties cafe, french toast cafe, cafe eggs benedict, avocado toast cafe, cafe fruit platter, granola bowl cafe, cafe oatmeal bar, cafe breakfast wrap, cafe lunch boxes, cafe meal prep, cafe kids menu, cafe senior menu, cafe dietary guide, cafe allergen list, cafe nutrition facts, cafe calorie guide, cafe ingredient list, cafe sourcing guide, cafe supplier list, cafe vendor guide, cafe cost analysis, cafe profit margins, cafe employee schedule, cafe training manual, cafe safety guide, cafe covid protocol, cafe health code, cafe inspection prep, cafe license guide, cafe insurance needs, cafe tax planning, cafe business loan, cafe renovation cost, cafe design guide, cafe seating plan, cafe kitchen layout, cafe storage solution, cafe equipment list, cafe supplier guide, cafe vendor contract, cafe delivery setup, cafe takeout system, cafe online ordering, cafe mobile app, cafe website design, cafe marketing plan, cafe social proof, cafe review management, cafe customer service, cafe complaint handling, cafe feedback system, cafe quality control, cafe inventory check, cafe waste reduction, cafe recycling plan, cafe composting guide, cafe green practices, cafe energy saving, cafe water conservation, cafe sustainable pack, cafe eco products, cafe local source, cafe seasonal menu, cafe special events, cafe holiday menu, cafe catering guide, cafe private events, cafe meeting space, cafe study area, cafe wifi setup, cafe music license, cafe entertainment, cafe trivia night, cafe open mic, cafe art display, cafe book corner, cafe game night, cafe poetry slam, cafe wine tasting, cafe beer selection, cafe cocktail menu, cafe mocktail list, cafe juice bar, cafe smoothie menu, cafe protein shake, cafe energy drinks, cafe bottled drinks, cafe water options, cafe window display, cafe sidewalk seating, cafe patio heaters, cafe outdoor lights, cafe umbrella setup, cafe garden design, cafe plant care, cafe interior plants, cafe wall decor, cafe lighting design, cafe furniture guide, cafe chair types, cafe table options, cafe counter height, cafe bar stools, cafe lounge areas, cafe reading nooks, cafe meeting pods, cafe power outlets, cafe internet speed, cafe printer setup, cafe laptop station, cafe phone charging, cafe digital menu, cafe qr code menu, cafe ordering tablet, cafe pos system, cafe payment methods, cafe credit cards, cafe mobile pay, cafe gift cards, cafe reward points, cafe membership plan, cafe birthday club, cafe senior discount, cafe student deals, cafe military discount, cafe happy hour deal, cafe early bird, cafe lunch special, cafe combo meals, cafe value menu, cafe family pack, cafe group order, cafe bulk order, cafe corporate rate, cafe event package, cafe meeting package, cafe birthday party, cafe wedding catering, cafe holiday party, cafe box lunch, cafe picnic basket, cafe breakfast box, cafe snack pack, cafe office coffee, cafe coffee cart, cafe mobile service, cafe delivery zone, cafe pickup window, cafe drive through, cafe curbside pickup, cafe contactless pay, cafe order ahead, cafe wait times, cafe rush hour, cafe peak times, cafe quiet hours, cafe closing time, cafe opening hours, cafe holiday hours, cafe seasonal hours, cafe staff break, cafe shift change, cafe cleaning time, cafe prep schedule, cafe baking schedule, cafe delivery times, cafe supplier days, cafe inventory days, cafe order cutoff, cafe freshness date, cafe expiry check, cafe storage temp, cafe fridge temp, cafe freezer log, cafe heat check, cafe food safety, cafe health permit, cafe fire safety, cafe emergency plan, cafe first aid, cafe security plan, cafe camera system, cafe alarm code, cafe key control, cafe safe access, cafe cash handle, cafe bank deposit, cafe petty cash, cafe daily sales, cafe weekly report, cafe monthly stats, cafe year review, cafe budget plan, cafe cost control, cafe price update, cafe menu costing, cafe recipe cost, cafe labor cost, cafe utility bill, cafe rent payment, cafe lease terms, cafe insurance due, cafe tax deadline, cafe audit prep, cafe compliance check, cafe health score, cafe safety audit, cafe quality check, cafe feedback form, cafe survey tool, cafe comment card, cafe online review, cafe social share, cafe photo policy, cafe wifi password, cafe guest log, cafe visit stats, cafe traffic count, cafe conversion rate, cafe return rate, cafe loyalty stats, cafe new customer, cafe regular guest, cafe vip member, cafe community post, cafe local event, cafe charity work, cafe sponsor local, cafe network group, cafe business meet, cafe equipment specs, cafe repair guide, cafe machine manual, cafe tool storage, cafe supply list, cafe order form, cafe vendor list, cafe price sheet, cafe menu template, cafe signage guide, cafe floor markers, cafe notice board, cafe staff board, cafe rule poster, cafe safety signs, cafe exit signs, cafe parking signs, cafe hours display, cafe menu board, cafe drink list, cafe food photos, cafe brand guide, cafe logo design, cafe color scheme, cafe uniform guide, cafe name badge, cafe staff photo, cafe team roster, cafe hire process, cafe job posting, cafe interview guide, cafe staff train, cafe onboarding, cafe handbook, cafe policy doc, cafe leave form, cafe pay scale, cafe bonus plan, cafe raise guide, cafe review form, cafe discipline, cafe grievance, cafe complaint box, cafe suggestion box, cafe team meet, cafe staff meal, cafe break room, cafe locker space, cafe storage bin, cafe supply shelf, cafe tool rack, cafe cup storage, cafe plate stack, cafe utensil box, cafe napkin hold, cafe straw caddy, cafe sugar caddy, cafe cream fridge, cafe syrup pump, cafe flavor rack, cafe tea drawer, cafe bean hopper, cafe grind bin, cafe filter box, cafe timer set, cafe scale guide, cafe measure cup, cafe shot glass, cafe steam wand, cafe drip tray, cafe knock box, cafe tamper mat, cafe group brush, cafe clean cloth, cafe sanitizer, cafe soap pump, cafe towel rack, cafe mop bucket, cafe broom set, cafe vacuum time, cafe window wash, cafe floor care, cafe wall clean, cafe vent check, cafe drain clean, cafe trash sort, cafe recycle bin, cafe compost bin, cafe waste guide, cafe pest check, cafe mouse trap, cafe bug spray, cafe air fresh, cafe plant feed, cafe light bulb, cafe battery stock, cafe tool kit, cafe fix guide, cafe paint touch, cafe scratch fix, cafe dent repair, cafe chair fix, cafe table level, cafe door swing, cafe lock check, cafe hinge oil, cafe wheel fix, cafe shelf mount, cafe rack install, cafe sign hang, cafe cord cover, cafe slip guard, cafe grip mat, cafe rain mat, cafe snow plan, cafe ice melt, cafe heat lamp, cafe fan guide, cafe temp set, cafe humid check, cafe air filter, cafe water test, cafe drain clean, cafe pipe check, cafe roof inspect, cafe wall test, cafe floor seal, cafe window seal, cafe door sweep, cafe draft stop, cafe noise check, cafe sound proof, cafe music zone, cafe quiet zone, cafe work zone, cafe serve zone, cafe prep zone, cafe clean zone, cafe storage zone, cafe staff zone, cafe guest zone, cafe flow chart, cafe task list, cafe duty check, cafe open list, cafe close list, cafe deep clean, cafe weekly task, cafe monthly job, cafe annual plan, cafe goals list, cafe growth plan, cafe trend watch, cafe market study, cafe comp check, cafe price scan, cafe menu update, cafe season plan, cafe event list, cafe promo plan, cafe ad budget, cafe media plan, cafe post schedule, cafe email list, cafe text alert, cafe app notice, cafe news blast, cafe blog post, cafe photo shoot, cafe video plan, cafe live stream, cafe story post, cafe share plan, cafe link build, cafe web rank, cafe search term, cafe local list, cafe map place, cafe review site, cafe rate watch, cafe guest post, cafe press kit, cafe brand book, cafe style guide, cafe font set, cafe image bank, cafe print file, cafe asset pack, cafe logo size, cafe card design, cafe menu art, cafe sign style, cafe pack guide, cafe box design, cafe bag print, cafe cup print, cafe label make, cafe sticker set, cafe stamp use, cafe tag design, cafe price tag, cafe shelf tag, cafe sale sign, cafe door sign, cafe wall art, cafe light plan, cafe plant map, cafe seat map, cafe flow path, cafe queue line, cafe wait area, cafe bus stop, cafe bike rack, cafe car park, cafe load zone, cafe trash zone, cafe store room, cafe cold room, cafe dry store, cafe tool room, cafe staff room, cafe meet room, cafe quiet pod, cafe phone booth, cafe work desk, cafe study spot, cafe game zone, cafe book shelf, cafe art wall, cafe music list, cafe sound check, cafe tech help, cafe fix line, cafe help desk, cafe train doc, cafe learn path, cafe skill test, cafe cert prep, cafe team build, cafe staff fun, cafe party plan, cafe team meal, cafe award show, cafe thank note, cafe gift list, cafe bonus time, cafe rest day, cafe leave plan, cafe sick cover, cafe temp staff, cafe peak hire, cafe quiet hire, cafe staff swap, cafe time trade, cafe split shift, cafe flex time, cafe week plan, cafe month view, cafe year goal, cafe five year, cafe term plan, cafe next step, cafe grow path, cafe branch plan, cafe fund seek, cafe cost save, cafe earn more, cafe waste less, cafe green plan, cafe earth help, cafe local good, cafe give back, cafe help out, cafe teach youth, cafe skill share, cafe art show, cafe book club, cafe game night, cafe quiz time, cafe music eve, cafe dance night, cafe food class, cafe cook show, cafe taste test, cafe bean roast, cafe grind demo, cafe pour show, cafe art teach, cafe milk art, cafe foam shape, cafe cup size, cafe temp guide, cafe blend mix, cafe roast time, cafe bean sort, cafe origin map, cafe taste wheel, cafe aroma chart, cafe body guide, cafe note card, cafe acid scale, cafe sweet spot, cafe drink temp, cafe ice rate, cafe water type, cafe filter grade, cafe grind size, cafe dose weight, cafe tamp press, cafe shot time, cafe flow rate, cafe crema check, cafe milk temp, cafe steam time, cafe froth test, cafe pour path, cafe art tool, cafe clean brush, cafe milk jug, cafe temp stick, cafe timer set, cafe scale type, cafe knock bar, cafe tamp mat, cafe group tool, cafe porta clean, cafe drain care, cafe water soft, cafe lime test, cafe soap type, cafe sani mix, cafe cloth sort, cafe rinse aid, cafe wash temp, cafe dry time, cafe store box, cafe date mark, cafe rotate stock, cafe temp log, cafe cool check, cafe heat test, cafe vent clean, cafe hood wash, cafe floor scrub, cafe wall wipe, cafe glass shine, cafe metal polish, cafe wood care, cafe cloth wash, cafe sponge change, cafe brush life, cafe tool care, cafe sharp keep, cafe rust stop, cafe metal coat, cafe wood seal, cafe paint fix, cafe scratch hide, cafe door fix, cafe lock care, cafe hinge oil, cafe wheel grease, cafe shelf care, cafe sign fix, cafe light change, cafe cord hide, cafe slip stop, cafe grip add, cafe drain flow, cafe pipe clear, cafe roof check, cafe wall seal, cafe floor grip, cafe noise cut, cafe sound fix, cafe zone mark, cafe flow sign, cafe task time, cafe duty split, cafe open step, cafe close task, cafe clean list, cafe deep task, cafe week work, cafe month task, cafe year plan, cafe goal set, cafe trend log, cafe cost list, cafe save plan, cafe earn goal, cafe waste cut, cafe green task, cafe help list, cafe skill note, cafe learn plan, cafe test time, cafe cert list, cafe team task, cafe staff note, cafe meet time, cafe break plan, cafe swap list, cafe time log, cafe work split, cafe hire need, cafe train time, cafe skill test, cafe path plan, cafe grow note, cafe lead task, cafe boss time, cafe own goal, cafe brand work, cafe style task, cafe look plan, cafe art need, cafe design time, cafe space use, cafe flow test, cafe move plan, cafe seat test, cafe walk path, cafe line flow, cafe wait time, cafe rush plan, cafe slow time, cafe peak work, cafe quiet task, cafe busy plan, cafe event time, cafe show list, cafe talk plan, cafe meet goal, cafe group task, cafe team time, cafe food plan, cafe drink list, cafe menu time, cafe price test, cafe cost goal, cafe sale plan, cafe buy list, cafe stock time, cafe use rate, cafe need list, cafe want plan, cafe must have, cafe tech need, cafe tool list, cafe fix plan, cafe care time, cafe clean goal, cafe neat keep, cafe safe plan, cafe risk list, cafe help time, cafe aid kit, cafe call list, cafe web plan, cafe app time, cafe post list, cafe share plan, cafe link time, cafe rank goal, cafe find ease, cafe look good, cafe feel right, cafe work well, cafe run smooth, cafe stay safe, cafe grow strong, cafe last long, cafe shine bright, cafe serve well, cafe help most, cafe give best, cafe do good, cafe be great, cafe aim high, cafe move up, cafe push on, cafe win more, cafe lead well, cafe teach all, cafe learn more, cafe share joy, cafe make fun, cafe stay true, cafe work hard, cafe play fair, cafe think big, cafe dream big, cafe start small, cafe grow fast";
    }

[LiveNathan]

Stack trace

{
  "keyword": "cafe card design",
  "industryAlignmentConfidence": 50
},
{
  "keyword": "cafe menu art",
  "industryAlignmentConfidence": 50
},
{
  "keyword": "cafe sign style",
  "industryAlignmentConfidence": 50
},
{
  "keyword": "cafe pack guide",
  "industryAlignmentConfidence": into class dev.nathanlively.adapter.out.ai.AiResponseKeywordsAligned

org.opentest4j.AssertionFailedError: Response should be convertible to AiResponseKeywordsAligned ==> Unexpected exception thrown: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input within/between Object entries
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 2861, column: 37] (through reference chain: dev.nathanlively.adapter.out.ai.AiResponseKeywordsAligned["keywordsWithThemeAndConfidence"]->java.util.HashSet[701])

at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:152)
at org.junit.jupiter.api.AssertDoesNotThrow.createAssertionFailedError(AssertDoesNotThrow.java:84)
at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:75)
at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:62)
at org.junit.jupiter.api.Assertions.assertDoesNotThrow(Assertions.java:3280)
at dev.nathanlively.adapter.out.ai.AiAdapterTest.given1000Keywords_whenProcessingWithStreamingChatClient_thenResponseIsSuccessful(AiAdapterTest.java:311)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:507)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1458)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2034)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189)

Caused by: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input within/between Object entries
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 2861, column: 37] (through reference chain: dev.nathanlively.adapter.out.ai.AiResponseKeywordsAligned["keywordsWithThemeAndConfidence"]->java.util.HashSet[701])
at org.springframework.ai.converter.BeanOutputConverter.convert(BeanOutputConverter.java:184)
at dev.nathanlively.adapter.out.ai.AiAdapterTest.lambda$given1000Keywords_whenProcessingWithStreamingChatClient_thenResponseIsSuccessful$3(AiAdapterTest.java:312)
at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:71)
... 9 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input within/between Object entries
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 2861, column: 37] (through reference chain: dev.nathanlively.adapter.out.ai.AiResponseKeywordsAligned["keywordsWithThemeAndConfidence"]->java.util.HashSet[701])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:401)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:372)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:381)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:246)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:543)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:585)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:447)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4917)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3860)
at org.springframework.ai.converter.BeanOutputConverter.convert(BeanOutputConverter.java:180)
... 11 more
Caused by: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input within/between Object entries
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 2861, column: 37]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:641)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipColon2(ReaderBasedJsonParser.java:2372)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipColon(ReaderBasedJsonParser.java:2287)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:712)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:432)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361)
... 23 more

[LiveNathan]

Additional findings that might help diagnose the root cause:

I initially thought this was a token limit issue since my first test showed truncation around the ~700th keyword. However, further testing revealed a more complex pattern:

1. Using batches of 40 keywords, I observed:

   • Consistent successful processing for batches 1-19 (~760 keywords total)
   • Each batch taking 15-16 seconds to process
   • Failure on batch 20, with JSON truncation at the 32nd keyword

2. Key observations:

   • Earlier batches successfully process all 40 keywords
   • JSON truncation occurs at different positions (previously around keyword 700, then 50, now at keyword 32 in batch 20)
   • Processing is consistent until sudden failure
   • Different truncation patterns in the JSON response

This suggests the issue might be related to:

• Rate limiting
• API timeouts
• Connection handling
• Response streaming behavior

Rather than a simple token limit issue, it seems there might be an underlying issue with how Spring AI handles extended streaming sessions or how it manages the connection over multiple batches.

Logs showing the pattern:

2025-02-11T19:59:47.679Z DEBUG 1 --- [onPool-worker-1] d.nathanlively.adapter.out.ai.AiAdapter  : Successfully processed batch 10/25
[...]
2025-02-11T20:02:10.120Z DEBUG 1 --- [onPool-worker-1] d.nathanlively.adapter.out.ai.AiAdapter  : Processing batch 20/25
2025-02-11T20:02:33.780Z ERROR 1 --- [edElastic-31813] o.s.ai.converter.BeanOutputConverter     : Could not parse the given text to the desired target type