Job Description

Company Description

SHAI (CY) Ltd is an AI-native automation platform. We build AI-first solutions across industries, starting from a deep first engagement in financial services (brokerage, fund administration, compliance) and expanding wherever we find a client and a problem worth solving.

 

About the Role

We are looking for an AI / Agent Engineer to join our team in Limassol. You will help build and maintain reliable, measurable, and controlled AI agents across the platform. This includes evaluating agent behavior before changes are released, monitoring and tracing agents in production, implementing guardrails, and keeping clear documentation of how decisions are made. This is an engineering role focused on the underlying AI agent runtime, reliability, monitoring, evaluation, and controls needed to operate agents safely and consistently at scale.

 

Responsibilities

Evaluation and regression

·         Build an evaluation harness: a curated suite of real cases per workflow that every agent must pass, run

·         automatically on each change, with pass rates tracked over time.

·         Establish a failure taxonomy for the agents (wrong tool call, hallucinated field, missed escalation, unsafe

·         action, formatting drift) and route fixes to root causes rather than symptoms.

·         Own the release gate: no agent change reaches the product without evaluation results.

 

Observability

·         Implement tracing of every agent run - prompts, tool calls, intermediate reasoning, outputs and latency - so a failure can be diagnosed in minutes and reproduced deterministically.

·         Feed agent traces into the platform's audit trail with provenance, satisfying the need to show who or what

·         took a decision and on what basis.

 

Guardrails and control

·         Design deterministic guardrails around the model: structured outputs with schema validation, allowed-action lists per role and workflow stage, confidence thresholds and mandatory human confirmation points.

·         Define, with the Technical Lead, which decisions agents may take autonomously, which they may only propose, and how exceptions and overrides are logged.

·         Manage model and prompt versioning so behaviour is reproducible and rollback is possible.

Architecture and cost

·         Contribute to the system architecture for the agent runtime: tool interfaces, retrieval, context management and state, so agents sit inside the platform's control points rather than around them.

·         Monitor and manage inference cost and latency per workflow; select models and patterns on evidence.

·         Coach the existing engineers and the AI/UX specialist in evaluation-driven development.

·         Perform other duties and responsibilities as required by Management to support the needs and objectives of the business.

 

Requirements

·         Good command of English.

·         Bachelor's degree in Computer Science, Software Engineering, or a related technical field.

·         1-3 years of software engineering with at least 1 year building LLM-based agents or assistants that reached production and were maintained there.

·         Have built evaluation pipelines and regression suites for model behaviour, and can explain how they measured improvement.

·         Hands-on experience with tracing and observability for LLM systems (OpenTelemetry-style tracing, LangSmith/Langfuse/Arize-type tooling or equivalent built in-house).

·         Strong experience with Python; comfortable with API design, structured outputs/JSON schema, tool-calling patterns and retrieval.

·         Can articulate a failure taxonomy for agents and the controls that address each class.

·         Good problem-solving and communication skills.

·         Ability to work with both technical and non-technical teams.

 

Nice to Have

·       System-architecture experience: has designed how an agent runtime integrates with a larger platform (state, tools, permissions, audit).

·         Experience in a regulated setting where AI outputs had to be explainable and auditable.

·         Familiarity with document-heavy workflows: OCR, extraction verification, e-mail-driven intake.

·         Experience managing inference cost and latency at production scale.

·         Experience taking AI projects from prototype to production.

·         Interest in keeping up with the latest AI developments

·         Russian or Greek is useful but not required.

We offer:

• Employment in AI-oriented firm

• Competitive salary

• 21 days annual leave, 5 sick days per year.

• Opportunity to work in an international environment with room for professional growth.

• Support in continuous development.

  

Working conditions:
Fully on-site

City: Limassol

Hours: 09:00 – 18:00 or 10:00 – 19:00

 

How to Apply

Please send CV at hr@shai.cy 

Soft Skills

Technical Skills

Job Categories

About Company