refactor: clarify agent pattern names
- Remove redundant workflow suffix from agentic pattern names and related classes - Update documentation to reflect pattern name changes
This commit is contained in:
Christian Tzolov
committed by
Christian Tzolov
Christian Tzolov
parent
1433ff7382
commit
9f8cb92aa9
19
agentic-patterns/evaluator-optimizer/.mvn/wrapper/maven-wrapper.properties
vendored
Normal file
19
agentic-patterns/evaluator-optimizer/.mvn/wrapper/maven-wrapper.properties
vendored
Normal file
@@ -0,0 +1,19 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
wrapperVersion=3.3.2
|
||||
distributionType=only-script
|
||||
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.9/apache-maven-3.9.9-bin.zip
|
||||
126
agentic-patterns/evaluator-optimizer/README.md
Normal file
126
agentic-patterns/evaluator-optimizer/README.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# Evaluator-Optimizer Pattern
|
||||
|
||||
This project demonstrates the Evaluator-Optimizer pattern for building effective LLM-based systems, as described in [Anthropic's research on building effective agents](https://www.anthropic.com/research/building-effective-agents).
|
||||
|
||||
|
||||
|
||||
## Overview
|
||||
|
||||
The Evaluator-Optimizer pattern implements a dual-LLM process where one model generates responses while another provides evaluation and feedback in an iterative loop, similar to a human writer's iterative refinement process. The pattern consists of two main components:
|
||||
|
||||
- **Generator LLM**: Produces initial responses and refines them based on feedback
|
||||
- **Evaluator LLM**: Analyzes responses and provides detailed feedback for improvement
|
||||
|
||||
## How It Works
|
||||
|
||||
1. The generator LLM produces an initial solution for the given task
|
||||
2. The evaluator LLM assesses the solution against quality criteria
|
||||
3. If the solution passes evaluation, it's returned as the final result
|
||||
4. If improvements are needed, feedback is incorporated into a new generation cycle
|
||||
5. The process repeats until a satisfactory solution is achieved
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Task[Task Input] --> G[Generator LLM]
|
||||
G --> E[Evaluator LLM]
|
||||
E -->|PASS| Out[Final Output]
|
||||
E -->|NEEDS_IMPROVEMENT| F[Feedback]
|
||||
F --> G
|
||||
```
|
||||
|
||||
## When to Use
|
||||
|
||||
This pattern is particularly effective when:
|
||||
|
||||
- Clear evaluation criteria exist for assessing response quality
|
||||
- Iterative refinement provides measurable value to the output
|
||||
- Tasks benefit from multiple rounds of critique and improvement
|
||||
|
||||
### Example Applications
|
||||
|
||||
- Literary translation requiring capture of subtle nuances
|
||||
- Complex search tasks needing multiple rounds of searching and analysis
|
||||
- Code generation where quality can be improved through systematic review
|
||||
- Content creation requiring multiple drafts and specific improvements
|
||||
|
||||
## Implementation
|
||||
|
||||
The implementation uses Spring AI's ChatClient for LLM interactions and consists of:
|
||||
|
||||
```java
|
||||
public class EvaluatorOptimizer {
|
||||
public RefinedResponse loop(String task) {
|
||||
// 1. Generate initial solution
|
||||
Generation generation = generate(task, context);
|
||||
|
||||
// 2. Evaluate the solution
|
||||
EvaluationResponse evaluation = evaluate(generation.response(), task);
|
||||
|
||||
// 3. If PASS, return solution
|
||||
// 4. If NEEDS_IMPROVEMENT, incorporate feedback and generate new solution
|
||||
// 5. Repeat until satisfactory
|
||||
return new RefinedResponse(finalSolution, chainOfThought);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Usage Example
|
||||
|
||||
```java
|
||||
ChatClient chatClient = // ... initialize chat client
|
||||
EvaluatorOptimizer agent = new EvaluatorOptimizer(chatClient);
|
||||
|
||||
// Process a task
|
||||
RefinedResponse response = agent.loop(
|
||||
"Create a Java class implementing a thread-safe counter"
|
||||
);
|
||||
|
||||
// Access results
|
||||
System.out.println("Final Solution: " + response.solution());
|
||||
System.out.println("Evolution: " + response.chainOfThought());
|
||||
```
|
||||
|
||||
## Customization
|
||||
|
||||
The pattern can be customized through:
|
||||
|
||||
1. **Custom Prompts**: Provide specialized prompts for generator and evaluator
|
||||
```java
|
||||
agent = new EvaluatorOptimizer(
|
||||
chatClient,
|
||||
customGeneratorPrompt,
|
||||
customEvaluatorPrompt
|
||||
);
|
||||
```
|
||||
|
||||
2. **Default Templates**: Modify the default prompts for common use cases
|
||||
- `DEFAULT_GENERATOR_PROMPT`: Template for solution generation
|
||||
- `DEFAULT_EVALUATOR_PROMPT`: Template for solution evaluation
|
||||
|
||||
## Response Formats
|
||||
|
||||
### Generation Response
|
||||
```json
|
||||
{
|
||||
"thoughts": "Brief description of approach",
|
||||
"response": "Actual solution content"
|
||||
}
|
||||
```
|
||||
|
||||
### Evaluation Response
|
||||
```json
|
||||
{
|
||||
"evaluation": "PASS|NEEDS_IMPROVEMENT|FAIL",
|
||||
"feedback": "Detailed feedback for improvement"
|
||||
}
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Spring AI
|
||||
- Spring Boot
|
||||
- Java 17 or later
|
||||
|
||||
## References
|
||||
|
||||
- [Building Effective Agents (Anthropic Research)](https://www.anthropic.com/research/building-effective-agents)
|
||||
259
agentic-patterns/evaluator-optimizer/mvnw
vendored
Executable file
259
agentic-patterns/evaluator-optimizer/mvnw
vendored
Executable file
@@ -0,0 +1,259 @@
|
||||
#!/bin/sh
|
||||
# ----------------------------------------------------------------------------
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Apache Maven Wrapper startup batch script, version 3.3.2
|
||||
#
|
||||
# Optional ENV vars
|
||||
# -----------------
|
||||
# JAVA_HOME - location of a JDK home dir, required when download maven via java source
|
||||
# MVNW_REPOURL - repo url base for downloading maven distribution
|
||||
# MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven
|
||||
# MVNW_VERBOSE - true: enable verbose log; debug: trace the mvnw script; others: silence the output
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
set -euf
|
||||
[ "${MVNW_VERBOSE-}" != debug ] || set -x
|
||||
|
||||
# OS specific support.
|
||||
native_path() { printf %s\\n "$1"; }
|
||||
case "$(uname)" in
|
||||
CYGWIN* | MINGW*)
|
||||
[ -z "${JAVA_HOME-}" ] || JAVA_HOME="$(cygpath --unix "$JAVA_HOME")"
|
||||
native_path() { cygpath --path --windows "$1"; }
|
||||
;;
|
||||
esac
|
||||
|
||||
# set JAVACMD and JAVACCMD
|
||||
set_java_home() {
|
||||
# For Cygwin and MinGW, ensure paths are in Unix format before anything is touched
|
||||
if [ -n "${JAVA_HOME-}" ]; then
|
||||
if [ -x "$JAVA_HOME/jre/sh/java" ]; then
|
||||
# IBM's JDK on AIX uses strange locations for the executables
|
||||
JAVACMD="$JAVA_HOME/jre/sh/java"
|
||||
JAVACCMD="$JAVA_HOME/jre/sh/javac"
|
||||
else
|
||||
JAVACMD="$JAVA_HOME/bin/java"
|
||||
JAVACCMD="$JAVA_HOME/bin/javac"
|
||||
|
||||
if [ ! -x "$JAVACMD" ] || [ ! -x "$JAVACCMD" ]; then
|
||||
echo "The JAVA_HOME environment variable is not defined correctly, so mvnw cannot run." >&2
|
||||
echo "JAVA_HOME is set to \"$JAVA_HOME\", but \"\$JAVA_HOME/bin/java\" or \"\$JAVA_HOME/bin/javac\" does not exist." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
else
|
||||
JAVACMD="$(
|
||||
'set' +e
|
||||
'unset' -f command 2>/dev/null
|
||||
'command' -v java
|
||||
)" || :
|
||||
JAVACCMD="$(
|
||||
'set' +e
|
||||
'unset' -f command 2>/dev/null
|
||||
'command' -v javac
|
||||
)" || :
|
||||
|
||||
if [ ! -x "${JAVACMD-}" ] || [ ! -x "${JAVACCMD-}" ]; then
|
||||
echo "The java/javac command does not exist in PATH nor is JAVA_HOME set, so mvnw cannot run." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# hash string like Java String::hashCode
|
||||
hash_string() {
|
||||
str="${1:-}" h=0
|
||||
while [ -n "$str" ]; do
|
||||
char="${str%"${str#?}"}"
|
||||
h=$(((h * 31 + $(LC_CTYPE=C printf %d "'$char")) % 4294967296))
|
||||
str="${str#?}"
|
||||
done
|
||||
printf %x\\n $h
|
||||
}
|
||||
|
||||
verbose() { :; }
|
||||
[ "${MVNW_VERBOSE-}" != true ] || verbose() { printf %s\\n "${1-}"; }
|
||||
|
||||
die() {
|
||||
printf %s\\n "$1" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
trim() {
|
||||
# MWRAPPER-139:
|
||||
# Trims trailing and leading whitespace, carriage returns, tabs, and linefeeds.
|
||||
# Needed for removing poorly interpreted newline sequences when running in more
|
||||
# exotic environments such as mingw bash on Windows.
|
||||
printf "%s" "${1}" | tr -d '[:space:]'
|
||||
}
|
||||
|
||||
# parse distributionUrl and optional distributionSha256Sum, requires .mvn/wrapper/maven-wrapper.properties
|
||||
while IFS="=" read -r key value; do
|
||||
case "${key-}" in
|
||||
distributionUrl) distributionUrl=$(trim "${value-}") ;;
|
||||
distributionSha256Sum) distributionSha256Sum=$(trim "${value-}") ;;
|
||||
esac
|
||||
done <"${0%/*}/.mvn/wrapper/maven-wrapper.properties"
|
||||
[ -n "${distributionUrl-}" ] || die "cannot read distributionUrl property in ${0%/*}/.mvn/wrapper/maven-wrapper.properties"
|
||||
|
||||
case "${distributionUrl##*/}" in
|
||||
maven-mvnd-*bin.*)
|
||||
MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/
|
||||
case "${PROCESSOR_ARCHITECTURE-}${PROCESSOR_ARCHITEW6432-}:$(uname -a)" in
|
||||
*AMD64:CYGWIN* | *AMD64:MINGW*) distributionPlatform=windows-amd64 ;;
|
||||
:Darwin*x86_64) distributionPlatform=darwin-amd64 ;;
|
||||
:Darwin*arm64) distributionPlatform=darwin-aarch64 ;;
|
||||
:Linux*x86_64*) distributionPlatform=linux-amd64 ;;
|
||||
*)
|
||||
echo "Cannot detect native platform for mvnd on $(uname)-$(uname -m), use pure java version" >&2
|
||||
distributionPlatform=linux-amd64
|
||||
;;
|
||||
esac
|
||||
distributionUrl="${distributionUrl%-bin.*}-$distributionPlatform.zip"
|
||||
;;
|
||||
maven-mvnd-*) MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ ;;
|
||||
*) MVN_CMD="mvn${0##*/mvnw}" _MVNW_REPO_PATTERN=/org/apache/maven/ ;;
|
||||
esac
|
||||
|
||||
# apply MVNW_REPOURL and calculate MAVEN_HOME
|
||||
# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-<version>,maven-mvnd-<version>-<platform>}/<hash>
|
||||
[ -z "${MVNW_REPOURL-}" ] || distributionUrl="$MVNW_REPOURL$_MVNW_REPO_PATTERN${distributionUrl#*"$_MVNW_REPO_PATTERN"}"
|
||||
distributionUrlName="${distributionUrl##*/}"
|
||||
distributionUrlNameMain="${distributionUrlName%.*}"
|
||||
distributionUrlNameMain="${distributionUrlNameMain%-bin}"
|
||||
MAVEN_USER_HOME="${MAVEN_USER_HOME:-${HOME}/.m2}"
|
||||
MAVEN_HOME="${MAVEN_USER_HOME}/wrapper/dists/${distributionUrlNameMain-}/$(hash_string "$distributionUrl")"
|
||||
|
||||
exec_maven() {
|
||||
unset MVNW_VERBOSE MVNW_USERNAME MVNW_PASSWORD MVNW_REPOURL || :
|
||||
exec "$MAVEN_HOME/bin/$MVN_CMD" "$@" || die "cannot exec $MAVEN_HOME/bin/$MVN_CMD"
|
||||
}
|
||||
|
||||
if [ -d "$MAVEN_HOME" ]; then
|
||||
verbose "found existing MAVEN_HOME at $MAVEN_HOME"
|
||||
exec_maven "$@"
|
||||
fi
|
||||
|
||||
case "${distributionUrl-}" in
|
||||
*?-bin.zip | *?maven-mvnd-?*-?*.zip) ;;
|
||||
*) die "distributionUrl is not valid, must match *-bin.zip or maven-mvnd-*.zip, but found '${distributionUrl-}'" ;;
|
||||
esac
|
||||
|
||||
# prepare tmp dir
|
||||
if TMP_DOWNLOAD_DIR="$(mktemp -d)" && [ -d "$TMP_DOWNLOAD_DIR" ]; then
|
||||
clean() { rm -rf -- "$TMP_DOWNLOAD_DIR"; }
|
||||
trap clean HUP INT TERM EXIT
|
||||
else
|
||||
die "cannot create temp dir"
|
||||
fi
|
||||
|
||||
mkdir -p -- "${MAVEN_HOME%/*}"
|
||||
|
||||
# Download and Install Apache Maven
|
||||
verbose "Couldn't find MAVEN_HOME, downloading and installing it ..."
|
||||
verbose "Downloading from: $distributionUrl"
|
||||
verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName"
|
||||
|
||||
# select .zip or .tar.gz
|
||||
if ! command -v unzip >/dev/null; then
|
||||
distributionUrl="${distributionUrl%.zip}.tar.gz"
|
||||
distributionUrlName="${distributionUrl##*/}"
|
||||
fi
|
||||
|
||||
# verbose opt
|
||||
__MVNW_QUIET_WGET=--quiet __MVNW_QUIET_CURL=--silent __MVNW_QUIET_UNZIP=-q __MVNW_QUIET_TAR=''
|
||||
[ "${MVNW_VERBOSE-}" != true ] || __MVNW_QUIET_WGET='' __MVNW_QUIET_CURL='' __MVNW_QUIET_UNZIP='' __MVNW_QUIET_TAR=v
|
||||
|
||||
# normalize http auth
|
||||
case "${MVNW_PASSWORD:+has-password}" in
|
||||
'') MVNW_USERNAME='' MVNW_PASSWORD='' ;;
|
||||
has-password) [ -n "${MVNW_USERNAME-}" ] || MVNW_USERNAME='' MVNW_PASSWORD='' ;;
|
||||
esac
|
||||
|
||||
if [ -z "${MVNW_USERNAME-}" ] && command -v wget >/dev/null; then
|
||||
verbose "Found wget ... using wget"
|
||||
wget ${__MVNW_QUIET_WGET:+"$__MVNW_QUIET_WGET"} "$distributionUrl" -O "$TMP_DOWNLOAD_DIR/$distributionUrlName" || die "wget: Failed to fetch $distributionUrl"
|
||||
elif [ -z "${MVNW_USERNAME-}" ] && command -v curl >/dev/null; then
|
||||
verbose "Found curl ... using curl"
|
||||
curl ${__MVNW_QUIET_CURL:+"$__MVNW_QUIET_CURL"} -f -L -o "$TMP_DOWNLOAD_DIR/$distributionUrlName" "$distributionUrl" || die "curl: Failed to fetch $distributionUrl"
|
||||
elif set_java_home; then
|
||||
verbose "Falling back to use Java to download"
|
||||
javaSource="$TMP_DOWNLOAD_DIR/Downloader.java"
|
||||
targetZip="$TMP_DOWNLOAD_DIR/$distributionUrlName"
|
||||
cat >"$javaSource" <<-END
|
||||
public class Downloader extends java.net.Authenticator
|
||||
{
|
||||
protected java.net.PasswordAuthentication getPasswordAuthentication()
|
||||
{
|
||||
return new java.net.PasswordAuthentication( System.getenv( "MVNW_USERNAME" ), System.getenv( "MVNW_PASSWORD" ).toCharArray() );
|
||||
}
|
||||
public static void main( String[] args ) throws Exception
|
||||
{
|
||||
setDefault( new Downloader() );
|
||||
java.nio.file.Files.copy( java.net.URI.create( args[0] ).toURL().openStream(), java.nio.file.Paths.get( args[1] ).toAbsolutePath().normalize() );
|
||||
}
|
||||
}
|
||||
END
|
||||
# For Cygwin/MinGW, switch paths to Windows format before running javac and java
|
||||
verbose " - Compiling Downloader.java ..."
|
||||
"$(native_path "$JAVACCMD")" "$(native_path "$javaSource")" || die "Failed to compile Downloader.java"
|
||||
verbose " - Running Downloader.java ..."
|
||||
"$(native_path "$JAVACMD")" -cp "$(native_path "$TMP_DOWNLOAD_DIR")" Downloader "$distributionUrl" "$(native_path "$targetZip")"
|
||||
fi
|
||||
|
||||
# If specified, validate the SHA-256 sum of the Maven distribution zip file
|
||||
if [ -n "${distributionSha256Sum-}" ]; then
|
||||
distributionSha256Result=false
|
||||
if [ "$MVN_CMD" = mvnd.sh ]; then
|
||||
echo "Checksum validation is not supported for maven-mvnd." >&2
|
||||
echo "Please disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2
|
||||
exit 1
|
||||
elif command -v sha256sum >/dev/null; then
|
||||
if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | sha256sum -c >/dev/null 2>&1; then
|
||||
distributionSha256Result=true
|
||||
fi
|
||||
elif command -v shasum >/dev/null; then
|
||||
if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | shasum -a 256 -c >/dev/null 2>&1; then
|
||||
distributionSha256Result=true
|
||||
fi
|
||||
else
|
||||
echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available." >&2
|
||||
echo "Please install either command, or disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ $distributionSha256Result = false ]; then
|
||||
echo "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised." >&2
|
||||
echo "If you updated your Maven version, you need to update the specified distributionSha256Sum property." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# unzip and move
|
||||
if command -v unzip >/dev/null; then
|
||||
unzip ${__MVNW_QUIET_UNZIP:+"$__MVNW_QUIET_UNZIP"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -d "$TMP_DOWNLOAD_DIR" || die "failed to unzip"
|
||||
else
|
||||
tar xzf${__MVNW_QUIET_TAR:+"$__MVNW_QUIET_TAR"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -C "$TMP_DOWNLOAD_DIR" || die "failed to untar"
|
||||
fi
|
||||
printf %s\\n "$distributionUrl" >"$TMP_DOWNLOAD_DIR/$distributionUrlNameMain/mvnw.url"
|
||||
mv -- "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" "$MAVEN_HOME" || [ -d "$MAVEN_HOME" ] || die "fail to move MAVEN_HOME"
|
||||
|
||||
clean || :
|
||||
exec_maven "$@"
|
||||
149
agentic-patterns/evaluator-optimizer/mvnw.cmd
vendored
Normal file
149
agentic-patterns/evaluator-optimizer/mvnw.cmd
vendored
Normal file
@@ -0,0 +1,149 @@
|
||||
<# : batch portion
|
||||
@REM ----------------------------------------------------------------------------
|
||||
@REM Licensed to the Apache Software Foundation (ASF) under one
|
||||
@REM or more contributor license agreements. See the NOTICE file
|
||||
@REM distributed with this work for additional information
|
||||
@REM regarding copyright ownership. The ASF licenses this file
|
||||
@REM to you under the Apache License, Version 2.0 (the
|
||||
@REM "License"); you may not use this file except in compliance
|
||||
@REM with the License. You may obtain a copy of the License at
|
||||
@REM
|
||||
@REM http://www.apache.org/licenses/LICENSE-2.0
|
||||
@REM
|
||||
@REM Unless required by applicable law or agreed to in writing,
|
||||
@REM software distributed under the License is distributed on an
|
||||
@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
@REM KIND, either express or implied. See the License for the
|
||||
@REM specific language governing permissions and limitations
|
||||
@REM under the License.
|
||||
@REM ----------------------------------------------------------------------------
|
||||
|
||||
@REM ----------------------------------------------------------------------------
|
||||
@REM Apache Maven Wrapper startup batch script, version 3.3.2
|
||||
@REM
|
||||
@REM Optional ENV vars
|
||||
@REM MVNW_REPOURL - repo url base for downloading maven distribution
|
||||
@REM MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven
|
||||
@REM MVNW_VERBOSE - true: enable verbose log; others: silence the output
|
||||
@REM ----------------------------------------------------------------------------
|
||||
|
||||
@IF "%__MVNW_ARG0_NAME__%"=="" (SET __MVNW_ARG0_NAME__=%~nx0)
|
||||
@SET __MVNW_CMD__=
|
||||
@SET __MVNW_ERROR__=
|
||||
@SET __MVNW_PSMODULEP_SAVE=%PSModulePath%
|
||||
@SET PSModulePath=
|
||||
@FOR /F "usebackq tokens=1* delims==" %%A IN (`powershell -noprofile "& {$scriptDir='%~dp0'; $script='%__MVNW_ARG0_NAME__%'; icm -ScriptBlock ([Scriptblock]::Create((Get-Content -Raw '%~f0'))) -NoNewScope}"`) DO @(
|
||||
IF "%%A"=="MVN_CMD" (set __MVNW_CMD__=%%B) ELSE IF "%%B"=="" (echo %%A) ELSE (echo %%A=%%B)
|
||||
)
|
||||
@SET PSModulePath=%__MVNW_PSMODULEP_SAVE%
|
||||
@SET __MVNW_PSMODULEP_SAVE=
|
||||
@SET __MVNW_ARG0_NAME__=
|
||||
@SET MVNW_USERNAME=
|
||||
@SET MVNW_PASSWORD=
|
||||
@IF NOT "%__MVNW_CMD__%"=="" (%__MVNW_CMD__% %*)
|
||||
@echo Cannot start maven from wrapper >&2 && exit /b 1
|
||||
@GOTO :EOF
|
||||
: end batch / begin powershell #>
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
if ($env:MVNW_VERBOSE -eq "true") {
|
||||
$VerbosePreference = "Continue"
|
||||
}
|
||||
|
||||
# calculate distributionUrl, requires .mvn/wrapper/maven-wrapper.properties
|
||||
$distributionUrl = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionUrl
|
||||
if (!$distributionUrl) {
|
||||
Write-Error "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties"
|
||||
}
|
||||
|
||||
switch -wildcard -casesensitive ( $($distributionUrl -replace '^.*/','') ) {
|
||||
"maven-mvnd-*" {
|
||||
$USE_MVND = $true
|
||||
$distributionUrl = $distributionUrl -replace '-bin\.[^.]*$',"-windows-amd64.zip"
|
||||
$MVN_CMD = "mvnd.cmd"
|
||||
break
|
||||
}
|
||||
default {
|
||||
$USE_MVND = $false
|
||||
$MVN_CMD = $script -replace '^mvnw','mvn'
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
# apply MVNW_REPOURL and calculate MAVEN_HOME
|
||||
# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-<version>,maven-mvnd-<version>-<platform>}/<hash>
|
||||
if ($env:MVNW_REPOURL) {
|
||||
$MVNW_REPO_PATTERN = if ($USE_MVND) { "/org/apache/maven/" } else { "/maven/mvnd/" }
|
||||
$distributionUrl = "$env:MVNW_REPOURL$MVNW_REPO_PATTERN$($distributionUrl -replace '^.*'+$MVNW_REPO_PATTERN,'')"
|
||||
}
|
||||
$distributionUrlName = $distributionUrl -replace '^.*/',''
|
||||
$distributionUrlNameMain = $distributionUrlName -replace '\.[^.]*$','' -replace '-bin$',''
|
||||
$MAVEN_HOME_PARENT = "$HOME/.m2/wrapper/dists/$distributionUrlNameMain"
|
||||
if ($env:MAVEN_USER_HOME) {
|
||||
$MAVEN_HOME_PARENT = "$env:MAVEN_USER_HOME/wrapper/dists/$distributionUrlNameMain"
|
||||
}
|
||||
$MAVEN_HOME_NAME = ([System.Security.Cryptography.MD5]::Create().ComputeHash([byte[]][char[]]$distributionUrl) | ForEach-Object {$_.ToString("x2")}) -join ''
|
||||
$MAVEN_HOME = "$MAVEN_HOME_PARENT/$MAVEN_HOME_NAME"
|
||||
|
||||
if (Test-Path -Path "$MAVEN_HOME" -PathType Container) {
|
||||
Write-Verbose "found existing MAVEN_HOME at $MAVEN_HOME"
|
||||
Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD"
|
||||
exit $?
|
||||
}
|
||||
|
||||
if (! $distributionUrlNameMain -or ($distributionUrlName -eq $distributionUrlNameMain)) {
|
||||
Write-Error "distributionUrl is not valid, must end with *-bin.zip, but found $distributionUrl"
|
||||
}
|
||||
|
||||
# prepare tmp dir
|
||||
$TMP_DOWNLOAD_DIR_HOLDER = New-TemporaryFile
|
||||
$TMP_DOWNLOAD_DIR = New-Item -Itemtype Directory -Path "$TMP_DOWNLOAD_DIR_HOLDER.dir"
|
||||
$TMP_DOWNLOAD_DIR_HOLDER.Delete() | Out-Null
|
||||
trap {
|
||||
if ($TMP_DOWNLOAD_DIR.Exists) {
|
||||
try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null }
|
||||
catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" }
|
||||
}
|
||||
}
|
||||
|
||||
New-Item -Itemtype Directory -Path "$MAVEN_HOME_PARENT" -Force | Out-Null
|
||||
|
||||
# Download and Install Apache Maven
|
||||
Write-Verbose "Couldn't find MAVEN_HOME, downloading and installing it ..."
|
||||
Write-Verbose "Downloading from: $distributionUrl"
|
||||
Write-Verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName"
|
||||
|
||||
$webclient = New-Object System.Net.WebClient
|
||||
if ($env:MVNW_USERNAME -and $env:MVNW_PASSWORD) {
|
||||
$webclient.Credentials = New-Object System.Net.NetworkCredential($env:MVNW_USERNAME, $env:MVNW_PASSWORD)
|
||||
}
|
||||
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
|
||||
$webclient.DownloadFile($distributionUrl, "$TMP_DOWNLOAD_DIR/$distributionUrlName") | Out-Null
|
||||
|
||||
# If specified, validate the SHA-256 sum of the Maven distribution zip file
|
||||
$distributionSha256Sum = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionSha256Sum
|
||||
if ($distributionSha256Sum) {
|
||||
if ($USE_MVND) {
|
||||
Write-Error "Checksum validation is not supported for maven-mvnd. `nPlease disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties."
|
||||
}
|
||||
Import-Module $PSHOME\Modules\Microsoft.PowerShell.Utility -Function Get-FileHash
|
||||
if ((Get-FileHash "$TMP_DOWNLOAD_DIR/$distributionUrlName" -Algorithm SHA256).Hash.ToLower() -ne $distributionSha256Sum) {
|
||||
Write-Error "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised. If you updated your Maven version, you need to update the specified distributionSha256Sum property."
|
||||
}
|
||||
}
|
||||
|
||||
# unzip and move
|
||||
Expand-Archive "$TMP_DOWNLOAD_DIR/$distributionUrlName" -DestinationPath "$TMP_DOWNLOAD_DIR" | Out-Null
|
||||
Rename-Item -Path "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" -NewName $MAVEN_HOME_NAME | Out-Null
|
||||
try {
|
||||
Move-Item -Path "$TMP_DOWNLOAD_DIR/$MAVEN_HOME_NAME" -Destination $MAVEN_HOME_PARENT | Out-Null
|
||||
} catch {
|
||||
if (! (Test-Path -Path "$MAVEN_HOME" -PathType Container)) {
|
||||
Write-Error "fail to move MAVEN_HOME"
|
||||
}
|
||||
} finally {
|
||||
try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null }
|
||||
catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" }
|
||||
}
|
||||
|
||||
Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD"
|
||||
82
agentic-patterns/evaluator-optimizer/pom.xml
Normal file
82
agentic-patterns/evaluator-optimizer/pom.xml
Normal file
@@ -0,0 +1,82 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>3.4.1</version>
|
||||
<relativePath /> <!-- lookup parent from repository -->
|
||||
</parent>
|
||||
<groupId>com.example.spring.ai</groupId>
|
||||
<artifactId>evaluator-optimizer</artifactId>
|
||||
<version>0.0.1-SNAPSHOT</version>
|
||||
<name>evaluator-optimizer</name>
|
||||
<description>Demo project for Spring Boot</description>
|
||||
<properties>
|
||||
<java.version>17</java.version>
|
||||
<spring-ai.version>1.0.0-M5</spring-ai.version>
|
||||
</properties>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-web</artifactId>
|
||||
</dependency>
|
||||
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- <dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-anthropic-spring-boot-starter</artifactId>
|
||||
</dependency> -->
|
||||
|
||||
|
||||
<!-- <dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
|
||||
</dependency> -->
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-bom</artifactId>
|
||||
<version>${spring-ai.version}</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
<repositories>
|
||||
<repository>
|
||||
<id>spring-milestones</id>
|
||||
<name>Spring Milestones</name>
|
||||
<url>https://repo.spring.io/milestone</url>
|
||||
<snapshots>
|
||||
<enabled>false</enabled>
|
||||
</snapshots>
|
||||
</repository>
|
||||
</repositories>
|
||||
|
||||
</project>
|
||||
@@ -0,0 +1,56 @@
|
||||
|
||||
/*
|
||||
* Copyright 2024 - 2024 the original author or authors.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.example.agentic;
|
||||
|
||||
import com.example.agentic.EvaluatorOptimizer.RefinedResponse;
|
||||
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.boot.CommandLineRunner;
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
|
||||
// ------------------------------------------------------------
|
||||
// EVALUATOR-OPTIMIZER
|
||||
// ------------------------------------------------------------
|
||||
|
||||
@SpringBootApplication
|
||||
public class Application {
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(Application.class, args);
|
||||
}
|
||||
|
||||
@Bean
|
||||
public CommandLineRunner commandLineRunner(ChatClient.Builder chatClientBuilder) {
|
||||
var chatClient = chatClientBuilder.build();
|
||||
return args -> {
|
||||
RefinedResponse refinedResponse = new EvaluatorOptimizer(chatClient).loop("""
|
||||
<user input>
|
||||
Implement a Stack in Java with:
|
||||
1. push(x)
|
||||
2. pop()
|
||||
3. getMin()
|
||||
All operations should be O(1).
|
||||
All inner fields should be private and when used should be prefixed with 'this.'.
|
||||
</user input>
|
||||
""");
|
||||
|
||||
System.out.println("FINAL OUTPUT:\n : " + refinedResponse);
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
/*
|
||||
* Copyright 2024 - 2024 the original author or authors.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.example.agentic;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.util.Assert;
|
||||
|
||||
/**
|
||||
* Workflow: <b>Evaluator-optimizer</b>
|
||||
* <p/>
|
||||
* Implements the Evaluator-Optimizer workflow pattern for Large Language Model
|
||||
* (LLM) interactions. This workflow orchestrates a dual-LLM process where one
|
||||
* model
|
||||
* generates responses while another provides evaluation and feedback in an
|
||||
* iterative loop,
|
||||
* similar to a human writer's iterative refinement process.
|
||||
*
|
||||
* <p>
|
||||
* The workflow consists of two main components:
|
||||
* <ul>
|
||||
* <li>A generator LLM that produces initial responses and refines them based on
|
||||
* feedback</li>
|
||||
* <li>An evaluator LLM that analyzes responses and provides detailed feedback
|
||||
* for improvement</li>
|
||||
* </ul>
|
||||
*
|
||||
* <b>Usage Criteria</b>
|
||||
* This workflow is particularly effective in scenarios that meet the following
|
||||
* conditions:
|
||||
* <ul>
|
||||
* <li>Clear evaluation criteria exist for assessing response quality</li>
|
||||
* <li>Iterative refinement provides measurable value to the output</li>
|
||||
* <li>The task benefits from multiple rounds of critique and improvement</li>
|
||||
* </ul>
|
||||
*
|
||||
* <b>Fitness Indicators</b>
|
||||
* Two key indicators suggest this workflow is appropriate:
|
||||
* <ul>
|
||||
* <li>LLM responses can be demonstrably improved when feedback is
|
||||
* articulated</li>
|
||||
* <li>The evaluator LLM can provide substantive and actionable feedback</li>
|
||||
* </ul>
|
||||
*
|
||||
* <b>Example Applications</b>
|
||||
* <ul>
|
||||
* <li>Literary translation requiring capture of subtle nuances through
|
||||
* iterative refinement</li>
|
||||
* <li>Complex search tasks needing multiple rounds of searching and
|
||||
* analysis</li>
|
||||
* <li>Code generation where quality can be improved through systematic
|
||||
* review</li>
|
||||
* <li>Content creation requiring multiple drafts and specific improvements</li>
|
||||
* </ul>
|
||||
*
|
||||
* @author Christian Tzolov
|
||||
* @see <a href=
|
||||
* "https://www.anthropic.com/research/building-effective-agents">Building
|
||||
* effective agents</a>
|
||||
*/
|
||||
@SuppressWarnings("null")
|
||||
public class EvaluatorOptimizer {
|
||||
|
||||
public static final String DEFAULT_GENERATOR_PROMPT = """
|
||||
Your goal is to complete the task based on the input. If there are feedback
|
||||
from your previous generations, you should reflect on them to improve your solution.
|
||||
|
||||
CRITICAL: Your response must be a SINGLE LINE of valid JSON with NO LINE BREAKS except those explicitly escaped with \\n.
|
||||
Here is the exact format to follow, including all quotes and braces:
|
||||
|
||||
{"thoughts":"Brief description here","response":"public class Example {\\n // Code here\\n}"}
|
||||
|
||||
Rules for the response field:
|
||||
1. ALL line breaks must use \\n
|
||||
2. ALL quotes must use \\"
|
||||
3. ALL backslashes must be doubled: \\
|
||||
4. NO actual line breaks or formatting - everything on one line
|
||||
5. NO tabs or special characters
|
||||
6. Java code must be complete and properly escaped
|
||||
|
||||
Example of properly formatted response:
|
||||
{"thoughts":"Implementing counter","response":"public class Counter {\\n private int count;\\n public Counter() {\\n count = 0;\\n }\\n public void increment() {\\n count++;\\n }\\n}"}
|
||||
|
||||
Follow this format EXACTLY - your response must be valid JSON on a single line.
|
||||
""";
|
||||
|
||||
public static final String DEFAULT_EVALUATOR_PROMPT = """
|
||||
Evaluate this code implementation for correctness, time complexity, and best practices.
|
||||
Ensure the code have proper javadoc documentation.
|
||||
Respond with EXACTLY this JSON format on a single line:
|
||||
|
||||
{"evaluation":"PASS, NEEDS_IMPROVEMENT, or FAIL", "feedback":"Your feedback here"}
|
||||
|
||||
The evaluation field must be one of: "PASS", "NEEDS_IMPROVEMENT", "FAIL"
|
||||
Use "PASS" only if all criteria are met with no improvements needed.
|
||||
""";
|
||||
|
||||
/**
|
||||
* Represents a solution generation step. Contains the model's thoughts and the
|
||||
* proposed solution.
|
||||
*
|
||||
* @param thoughts The model's understanding of the task and feedback
|
||||
* @param response The model's proposed solution
|
||||
*/
|
||||
public static record Generation(String thoughts, String response) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents an evaluation response. Contains the evaluation result and
|
||||
* detailed feedback.
|
||||
*
|
||||
* @param evaluation The evaluation result (PASS, NEEDS_IMPROVEMENT, or FAIL)
|
||||
* @param feedback Detailed feedback for improvement
|
||||
*/
|
||||
public static record EvaluationResponse(Evaluation evaluation, String feedback) {
|
||||
|
||||
public enum Evaluation {
|
||||
PASS, NEEDS_IMPROVEMENT, FAIL
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents the final refined response. Contains the final solution and the
|
||||
* chain of thought showing the evolution of the solution.
|
||||
*
|
||||
* @param solution The final solution
|
||||
* @param chainOfThought The chain of thought showing the evolution of the
|
||||
* solution
|
||||
*/
|
||||
public static record RefinedResponse(String solution, List<Generation> chainOfThought) {
|
||||
}
|
||||
|
||||
private final ChatClient chatClient;
|
||||
|
||||
private final String generatorPrompt;
|
||||
|
||||
private final String evaluatorPrompt;
|
||||
|
||||
public EvaluatorOptimizer(ChatClient chatClient) {
|
||||
this(chatClient, DEFAULT_GENERATOR_PROMPT, DEFAULT_EVALUATOR_PROMPT);
|
||||
}
|
||||
|
||||
public EvaluatorOptimizer(ChatClient chatClient, String generatorPrompt, String evaluatorPrompt) {
|
||||
Assert.notNull(chatClient, "ChatClient must not be null");
|
||||
Assert.hasText(generatorPrompt, "Generator prompt must not be empty");
|
||||
Assert.hasText(evaluatorPrompt, "Evaluator prompt must not be empty");
|
||||
|
||||
this.chatClient = chatClient;
|
||||
this.generatorPrompt = generatorPrompt;
|
||||
this.evaluatorPrompt = evaluatorPrompt;
|
||||
}
|
||||
|
||||
/**
|
||||
* Initiates the evaluator-optimizer workflow for a given task. This method
|
||||
* orchestrates the iterative process of generation and evaluation until a
|
||||
* satisfactory solution is reached.
|
||||
*
|
||||
* <p>
|
||||
* The workflow follows these steps:
|
||||
* </p>
|
||||
* <ol>
|
||||
* <li>Generate an initial solution</li>
|
||||
* <li>Evaluate the solution against quality criteria</li>
|
||||
* <li>If evaluation passes, return the solution</li>
|
||||
* <li>If evaluation indicates need for improvement, incorporate feedback and
|
||||
* generate new solution</li>
|
||||
* <li>Repeat steps 2-4 until a satisfactory solution is achieved</li>
|
||||
* </ol>
|
||||
*
|
||||
* @param task The task or problem to be solved through iterative refinement
|
||||
* @return A RefinedResponse containing the final solution and the chain of
|
||||
* thought
|
||||
* showing the evolution of the solution
|
||||
*/
|
||||
public RefinedResponse loop(String task) {
|
||||
List<String> memory = new ArrayList<>();
|
||||
List<Generation> chainOfThought = new ArrayList<>();
|
||||
|
||||
return loop(task, "", memory, chainOfThought);
|
||||
}
|
||||
|
||||
/**
|
||||
* Internal recursive implementation of the evaluator-optimizer loop. This
|
||||
* method
|
||||
* maintains the state of previous attempts and feedback while recursively
|
||||
* refining
|
||||
* the solution until it meets the evaluation criteria.
|
||||
*
|
||||
* @param task The original task to be solved
|
||||
* @param context Accumulated context including previous attempts and
|
||||
* feedback
|
||||
* @param memory List of previous solution attempts for reference
|
||||
* @param chainOfThought List tracking the evolution of solutions and reasoning
|
||||
* @return A RefinedResponse containing the final solution and complete solution
|
||||
* history
|
||||
*/
|
||||
private RefinedResponse loop(String task, String context, List<String> memory,
|
||||
List<Generation> chainOfThought) {
|
||||
|
||||
Generation generation = generate(task, context);
|
||||
memory.add(generation.response());
|
||||
chainOfThought.add(generation);
|
||||
|
||||
EvaluationResponse evaluationResponse = evalute(generation.response(), task);
|
||||
|
||||
if (evaluationResponse.evaluation().equals(EvaluationResponse.Evaluation.PASS)) {
|
||||
// Solution is accepted!
|
||||
return new RefinedResponse(generation.response(), chainOfThought);
|
||||
}
|
||||
|
||||
// Accumulated new context including the last and the previous attempts and
|
||||
// feedbacks.
|
||||
StringBuilder newContext = new StringBuilder();
|
||||
newContext.append("Previous attempts:");
|
||||
for (String m : memory) {
|
||||
newContext.append("\n- ").append(m);
|
||||
}
|
||||
newContext.append("\nFeedback: ").append(evaluationResponse.feedback());
|
||||
|
||||
return loop(task, newContext.toString(), memory, chainOfThought);
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates or refines a solution based on the given task and feedback context.
|
||||
* This method represents the generator component of the workflow, producing
|
||||
* responses that can be iteratively improved through evaluation feedback.
|
||||
*
|
||||
* @param task The primary task or problem to be solved
|
||||
* @param context Previous attempts and feedback for iterative improvement
|
||||
* @return A Generation containing the model's thoughts and proposed solution
|
||||
*/
|
||||
private Generation generate(String task, String context) {
|
||||
Generation generationResponse = chatClient.prompt()
|
||||
.user(u -> u.text("{prompt}\n{context}\nTask: {task}")
|
||||
.param("prompt", this.generatorPrompt)
|
||||
.param("context", context)
|
||||
.param("task", task))
|
||||
.call()
|
||||
.entity(Generation.class);
|
||||
|
||||
System.out.println(String.format("\n=== GENERATOR OUTPUT ===\nTHOUGHTS: %s\n\nRESPONSE:\n %s\n",
|
||||
generationResponse.thoughts(), generationResponse.response()));
|
||||
return generationResponse;
|
||||
}
|
||||
|
||||
/**
|
||||
* Evaluates if a solution meets the specified requirements and quality
|
||||
* criteria.
|
||||
* This method represents the evaluator component of the workflow, analyzing
|
||||
* solutions
|
||||
* and providing detailed feedback for further refinement until the desired
|
||||
* quality
|
||||
* level is reached.
|
||||
*
|
||||
* @param content The solution content to be evaluated
|
||||
* @param task The original task against which to evaluate the solution
|
||||
* @return An EvaluationResponse containing the evaluation result
|
||||
* (PASS/NEEDS_IMPROVEMENT/FAIL)
|
||||
* and detailed feedback for improvement
|
||||
*/
|
||||
private EvaluationResponse evalute(String content, String task) {
|
||||
|
||||
EvaluationResponse evaluationResponse = chatClient.prompt()
|
||||
.user(u -> u.text("{prompt}\nOriginal task: {task}\nContent to evaluate: {content}")
|
||||
.param("prompt", this.evaluatorPrompt)
|
||||
.param("task", task)
|
||||
.param("content", content))
|
||||
.call()
|
||||
.entity(EvaluationResponse.class);
|
||||
|
||||
System.out.println(String.format("\n=== EVALUATOR OUTPUT ===\nEVALUATION: %s\n\nFEEDBACK: %s\n",
|
||||
evaluationResponse.evaluation(), evaluationResponse.feedback()));
|
||||
return evaluationResponse;
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
spring.application.name=mcp
|
||||
spring.main.web-application-type=none
|
||||
|
||||
# OpenAI
|
||||
spring.ai.openai.api-key=${OPENAI_API_KEY}
|
||||
|
||||
# Anthropic
|
||||
spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}
|
||||
|
||||
# Ollama
|
||||
# spring.ai.ollama.chat.options.model=qwen2.5:latest
|
||||
spring.ai.ollama.chat.options.model=llama3.2:latest
|
||||
spring.ai.ollama.init.pull-model-strategy=always
|
||||
spring.ai.ollama.init.timeout=60s
|
||||
spring.ai.ollama.init.max-retries=2
|
||||
Reference in New Issue
Block a user