Vertex claude api can not stream output by using ai-chatbot template

I use nextjs-ai-chatbot(Next.js AI Chatbot) template ,and change the llm from openai to vertex claude api by using GitHub - nalaso/anthropic-vertex-ai: Vercel AI community package for using anthropic through vertex ai..

and then I can’t get stream output, I have tried many times, but still can’t stream output. I don’t know why. Has anyone encountered the same problem as me?

my core code is here:

console.log(model.apiIdentifier);
  const result = await streamText({
    model: modelInstance,
    system: modelId === 'claude-3-5-coder' ? CodePrompt : regularPrompt,
    messages: coreMessages,
    maxSteps: 5,

    onFinish: async ({ responseMessages }) => {
      if (session.user && session.user.id) {
        try {
          const responseMessagesWithoutIncompleteToolCalls =
            sanitizeResponseMessages(responseMessages);
          console.log("responseMessagesWithoutIncompleteToolCalls:",responseMessagesWithoutIncompleteToolCalls);
          await saveChat({
            id,
            messages: [
              ...coreMessages,
              ...responseMessagesWithoutIncompleteToolCalls,
            ],
            userId: session.user.id,
          });
        } catch (error) {
          console.error('Failed to save chat');
        }
      }
    },
  });
  for await (const textPart of result.textStream){
    console.log(textPart)
  }
  //console.log('data:', result.toDataStreamResponse({  data: streamingData,}))
  return result.toDataStreamResponse();
}

1 post - 1 participant

Read full topic