Prompt evaluation harness
Deliverable: A test set of at least fifty cases with scoring criteria, comparing three prompt strategies with measured quality differences rather than impressions.
Course
Generative AI
A short, applied track for people who already program — prompting with evaluation attached, retrieval systems, and the guardrails that separate a demo from something you can ship.
- Duration: 3 months
- Classroom · Online live
- Level: intermediate
What you will be able to do
- Write production-shaped prompts with an output contract rather than by trial and error
- Build an evaluation set before iterating, and measure whether a change improved anything
- Design and build a retrieval-augmented generation system over a private document set
- Choose chunking and retrieval strategies deliberately and justify the trade-offs
- Measure groundedness and detect hallucination rather than judging by impression
- Apply guardrails against prompt injection and unsafe output
- Reason about cost, latency and model selection for a given feature
Who this course is for
- Working developers, analysts or testers who can already write Python
- Graduates who have completed a Python or full stack course
- Product and technical staff who need to evaluate AI features rather than build them
- Not suitable as a first programming course
Syllabus
6 modules · 3 months
Module 1. How language models actually work
- Tokenisation, embeddings and what a vector represents
- Attention and the transformer, without the mathematics
- Pretraining, fine-tuning and instruction tuning
- Context windows, and why long context is not free
- Why models hallucinate, mechanically rather than mystically
Module 2. Prompting as engineering
- Instruction, context, examples and output contract
- Few-shot prompting and example selection
- Chain-of-thought and when it helps or hurts
- Structured output and function calling
- Prompt versioning and treating prompts as code
Module 3. Evaluation
- Building a test set before you start iterating
- Scoring rubrics and inter-rater agreement
- Automated evaluation and using a model as a judge
- Regression testing prompts as models change under you
- Reading a benchmark critically
Module 4. Retrieval-augmented generation
- What RAG solves and what it does not
- Chunking strategies and their effect on retrieval quality
- Embeddings and vector stores
- Hybrid retrieval and reranking
- Measuring groundedness and citation accuracy
- Diagnosing whether a bad answer was retrieval or generation
Module 5. Building and shipping
- Application structure around an LLM call
- Streaming, timeouts and graceful degradation
- Caching and cost control
- Rate limits, retries and failure handling
- A simple interface with Streamlit
Module 6. Safety and responsibility
- Prompt injection and why input sanitisation is not enough
- Guardrails, refusals and output filtering
- Privacy, data residency and what not to send to a hosted model
- Bias in outputs and in evaluation sets
- Monitoring an AI feature after release
Tools and technologies you will use
- Python
- Anthropic and OpenAI APIs
- Hugging Face Transformers
- LangChain
- FAISS and Chroma
- Streamlit
- Git
Projects you will build
Retrieval-augmented assistant
Deliverable: A working RAG application over a private document set, with a documented chunking and retrieval strategy, a groundedness evaluation, and an honest list of queries it still answers badly.
Structured extraction pipeline
Deliverable: A pipeline extracting structured records from unstructured documents, with schema validation, a measured accuracy figure and defined behaviour when extraction fails.
Guardrail and injection test
Deliverable: A set of adversarial inputs against your own application, with the failures found, the guardrails added, and evidence of what those guardrails do and do not stop.
Where this course can take you
- AI Engineer
- Applied AI Developer
- LLM Application Developer
- AI Product Engineer
- Automation Engineer
Duration, modes and fees
- Duration
- 3 months
- Delivery modes
- Classroom · Online live
- Fees
- Share your details for the current fee
- Fees vary by batch and delivery mode. Share your details and an advisor will confirm the current fee.
Placement assistance
Every student gets placement assistance — that is what 100% placement assistance means. It is support for all, not a job for all. We do not promise a specific salary, a specific number of interviews, or placement at any named company, and you should be wary of anyone who does.
What is included
- A place in the monthly placement drive, held every third Saturday
- The readiness programme every second Saturday — mock interviews and preparation
- CV review against the specific roles you are targeting
- Portfolio review, so your project work is presented the way a reviewer will read it
- Access to the vacancy pool employers send directly to the Samyak network
- Guidance on which roles realistically fit your background and which do not
- A place in the next drive, with coaching, if you are not selected in this one
What is not included
- Any guarantee of a job, an interview, or a particular salary
- Placement at a named or partner company
- Applying to jobs on your behalf
- Support before you have completed the course and its project work
- Visa, relocation or overseas placement assistance
The habit this course is really teaching
Build the test set before you build the feature.
Most people iterate on prompts by feel. They change a wording, the output looks better, they declare success. Then it behaves differently on the twentieth input and nobody knows why, because there was never a baseline.
Every applied module here requires an evaluation set first. By the end you will be able to say that a change improved groundedness on a fifty-case set — which is the difference between an opinion and a result, and it is the single most employable habit in applied AI right now.
Why this is not a beginner course
You need to program already. That is a firm prerequisite rather than a suggestion.
The reason is that the interesting work is not the model call — it is everything around it. Handling a timeout. Deciding what to do when retrieval returns nothing. Structuring the application so a prompt change does not require a rewrite. Debugging why extraction works on one document format and fails on another.
Someone without programming fundamentals can produce a working demo and cannot progress past it, which is a frustrating place to be and a crowded one.
RAG, and what it does not fix
Retrieval-augmented generation is presented almost everywhere as the answer to hallucination. It is not.
RAG grounds a model in documents you supply. It does not stop the model misreading them, combining two passages into a false claim, or answering confidently when retrieval returned nothing relevant. Those failure modes remain, and they are subtler because the output now cites a source.
Module four spends real time on diagnosing whether a bad answer came from retrieval or from generation, because the fixes are completely different and teams routinely spend weeks fixing the wrong one.
Honest scope
Three months makes you capable of building, evaluating and shipping LLM features, and of telling a colleague why a proposed approach will not work.
It does not make you a machine learning engineer, and it does not replace software engineering fundamentals if you want to work on production AI infrastructure. We say so on the first day.
Questions
Generative AI — frequently asked questions
How is this different from your artificial intelligence course?
The AI course is a six-month track that builds machine learning and deep learning foundations first, then covers generative AI on top. This is a focused three-month track that assumes you already program and goes straight to applied LLM work. Choose this if you want to build AI features soon; choose the AI course if you want the underlying foundations too.
Do I need machine learning knowledge before this course?
No, but you do need to program comfortably in Python. This track treats the model as a component you build around rather than something you train, which is what most applied AI work actually involves. If you want to train and evaluate models yourself, the AI or data science course is the right choice.
Will I learn to train my own large language model?
No, and no three-month course honestly can — pretraining a competitive model costs millions of dollars. You will learn to build applications around existing models, retrieve against private data, and evaluate output rigorously. That is what applied roles hire for.
Is prompt engineering a real skill or a passing fad?
The narrow version — collecting clever phrases — was always thin and is fading. The durable version is building an evaluation set, measuring whether a change helped, and designing systems that degrade sensibly when the model gets something wrong. That is engineering, and this course is built around it rather than around prompt lists.
Enquire about Generative AI
Three details is all we need. A course advisor will call you back.
Related courses
- Artificial Intelligence
Python, machine learning foundations and applied generative AI in one track — ending with a retrieval-augmented application you have built, evaluated and can explain end to end.
- Python Programming
Python taught as a working tool rather than a syntax tour — you finish able to read unfamiliar code, automate real tasks, call APIs and write tests that catch your own mistakes.
- Machine Learning
Classical machine learning done properly — feature engineering, honest evaluation, and the discipline to notice when a good score is an artefact rather than a result.
Next step
Talk to a course advisor
Tell us what you want to learn and we will help you pick the right course, batch and mode.