My Experience with AI-300 / OSAI

It was a Thursday of August when I decided to take on the exam for the OSAI certification, the latest cert from OffSec which focuses on vulnerable AI systems in a red-teaming context. Why Thursday specifically? I planned to finish everything before Friday night so that I could finish with some drinks with my friends. Did I manage to do it? Yes!

After 19 hours of basically-no-rest exam experience, I was able to finish 7 of the 8 machines required, obtaining a total of 85/100 points, enough to pass. I finally slept, and the next day I spent another 5 hours to prepare and finalize the report. After a day I got the confirmation of passing the exam. In this blog post I want to answer some useful questions related to the certification.

I hope you find it useful :)

Why did I tackle AI-300?#

The question of why I decided to take on the AI-300 is related to the recent years in my life. A year ago I decided to quit my main job as penetration tester in order to try and build a career as a freelance trainer teaching cybersecurity.

My main objective is actually much more broad than cybersecurity. I want to improve the ways in which we teach Computer Science in Italy and in the world. I decided to start in cybersecurity because in my old job I was doing a lot of training with various companies, and I realized that many companies, especially in Italy, still don't know much about cybersecurity topics such as secure coding, threat modeling, and in general how to think about security risks, vulnerabilities, threats and exploits in the context of the software development life-cycle. Also, cybersecurity is a great field to study, as it forces you to learn about how computers really work.

I opened a Partita Iva Forfettaria, which basically means doing freelance in Italy, and since then I've been spending most of my time finding new clients, preparing material for the trainings, recording lots and lots of teaching material, and I've opened an online teaching platform , which right now its only in Italian, but in the future I also want to make it for a more international / english audience. I leave some links here for those interested. Do not worry, it will come in english as well in the near future!

• https://esadecimale.it
• https://learn.esadecimale.it
• https://cyber.esadecimale.it

As I was searching for new clients, I saw that OffSec was searching for trainers in order to teach their classes. I saw the opportunity and took it. I wrote to them describing my experience. They liked it, and after a test lecture, where they wanted me to present something, they liked it and I got into their external contractor trainer program. This means that I'm not an OffSec employee, but an external contractor that they can call when they need to deliver a specific class for a specific certification. One thing they told me about their policy is that you cannot teach a class for a certification that you do not own. This makes sense: you cannot teach what you did not do.

Around a month ago they wrote to me, asking if I wanted to take their latest certification (AI-300/OSAI) in order to teach future classes on it. I took the opportunity as it came, put aside everything else I was doing, and I 100% focused on it. A few days ago I did the exam, and now I'm ready to help OffSec teach OSAI material around the world!

What's AI-300 all about?#

This is the official description from OffSec of AI-300

• https://www.offsec.com/courses/ai-300/

Gain hands-on expertise in AI cybersecurity, including attacking LLMs, multi-agent AI systems, RAG pipelines, embeddings, and AI infrastructure while building skills in securing AI, cloud security, and modern AI technologies

Having gone through the course, the challenge labs, and now the exam, I can confirm that this description is accurate. AI-300 is a particular beast, as it combines many different areas of knowledge together. While a good portion of the course is about the latest AI technologies and their vulnerabilities, it's not just generative AI. It also includes traditional attack vectors such as supply-chain attacks and attacks on development pipelines and infrastructures that are becoming ever more important.

This course is not about the maths behind generative AI and Large Language Models. There is only one module that goes more into such attacks (Attacking Embeddings), but other than that the course does not go inside the models nor inside the training process. We assume a company is using one or more already trained LLMs. The course is heavy on penetration testing. Its all about understanding practical risks and flaws that show up in systems that contain both traditional technologies as well as the latest generative AI technologies.

To build a good mental model for what OSAI is, start with OSCP. OSCP teaches you the basics of penetration testing in Linux, Windows and Active Directory. Take OSCP, but then expand it much more. Because now you have to also consider all the technologies used during development and used to make the model available for the other parts of the system. That is, all the AI infrastructured required to have the models actually do meaningful work in a digital system. Then you have to consider the supply-chain, an extremely important attack vector that is being exploited more and more as attackers are becoming aware of its weaknesses. Finally, we have to consider all the different ways in which LLMs are integrated in the modern SDLC (Software Development Life Cycle). Basically all the different technologies that were built in the past year on top of LLMs (RAG, MCP, agents, and so on). That's how you get OSAI.

$$\begin{split} \texttt{OSAI} = \texttt{OSCP} &+ \texttt{LLMs} \\ &+ \texttt{Technologies built on top of LLMs} \\ &+ \texttt{Technologies for Development} \\ &+ \texttt{AI Infrastructure} \\ &+ \texttt{Supply-chain attack vectors} \\ \end{split}$$

This is the list of modules present in the course:

1. Introduction to Red Teaming AI Systems
2. Reconnaissance for AI Targets
3. Attacking AI Agents
4. Attacking Multi-Agent Systems & Agent-to-Agent (A2A) Protocol
5. Exploiting RAG Pipelines
6. Attacking Embeddings
7. Attacking Model Context Protocol and Tool Surfaces
8. Supply Chain Attacks on AI/ML Systems
9. AI Infrastructure and Deployment Exploits
10. Threat Modeling for AI-Enabled Targets
11. Assembling The Pieces - Capstone Red Team Engagement

The most theoretical chapter is probably chapter 6, the one on attacking Embeddings. It is also the module you need the least in order to complete the exam in my opinion. The style is similar to the other OffSec certs.

Currently there are also 5 challenge labs. As always when it comes to OffSec certifications, the challenge labs are really important. You can use them to asses your level and if you are ready for the final exam.

1. Shadow Supply
2. Pipeline Breach
3. Iron Crown
4. Double Helix
5. Synthetic Siege

Between the textual content and the challenge labs, I prefered much more the challenge labs, because I am a practical guy and I learn much better by doing hard things. The labs showcase interesting scenarios I learned new things doing them. Consider also that during the labs there are also some evasion techniques that you need to use against Windows Defender. While it is not a crazy amount, and it is not hard to do, it's still there and it adds to the mix.

Having done the exam I can also say that the challenge labs are good mental models for the final exam. There is however a big difference between the labs and the final exam. Currently the challenge labs are structured in explicit steps. You start the lab and they give you a task, and then you have the next task and the next task. Basically its broken down in a linear path, task after task. This means you always know the next objective you need to reach. In the exam you lose this structure. In the exam you just see a lot of different services. It is up to you to understand where are the rabbit holes and where are the interesting surfaces to explore and attack. So be mindful to not rely too much on the task description, because they will not be there in the exam.

The OSAI Exam Structure#

The exam is made up of 10 servers, but not all of them are vulnerable:

• 8 servers are vulnerable.
• 2 servers are not vulnerable.

The eight vulnerable servers are divided as follows:

• Two chains, where each chain is made up of three servers
• There is a Domain Controller (DC) at the end of each chain
• There is a independent server that is not tied to any chain

There are two networks:

• A public network, that contains the two foothold
• A private network, that contains all the other machines

Initially you are given two public IPs. These represent the start of the two chains. Each IP has a foothold and you have to find it. At the end of each chain we find the same Domain Controller. This means that in terms of topology we find the following:

In terms of flags, there are three types of flags:

• $\texttt{ai_vector.txt} \rightarrow 15 \texttt{ points}$
• $\texttt{trad_vector.txt} \rightarrow 10 \texttt{ points}$
• $\texttt{proof.txt} \rightarrow 5 \texttt{ points}$

The proof flag is found in the DC and it can only be submitted once, even if you finish the two chains. The independent server always contains an ai vector flag, while the other machines can contain either a traditional vector flag or an ai vector flag. Which one you find depends on the vulnerabilities found in the machine.

In order to pass the exam you must obtain at least 75 points out of 100 points.

My Exam Experience#

The 15 minutes of proctoring setup went quickly and fine. You connect to a tailscale endpoint, start the lab and you can access two public IPs. Each IP has a foothold. You have to find it.

The countdown started at 13:00 Europe/Rome on Thursday 6 August.

I spent the first 2.5 hours to find the first foothold. The chain required to obtain it was not extremely complex, but it was made up of a bunch of different steps. There were multiple layers, some rabbit holes and some defenses. Here what I mention about the challenge labs is really important: in the exam you have to explore everything, and you have to understand when something is clearly a rabbit hole in order to focus on the actual exploitation path. Having understood it all, I was able to enter into the first machine.

After the first foothold I setup tunneling with chisel and I was able to continue to do the rest of the chain. Nothing was ever too quick, but I was making steady progress, and after a few hours I was able to finish the entire chain and enter into the Domain Controller. Roughly it was around 1 hour per machine in the chain, while the first foothold took longer. I also noticed at this point that each server had a big idea to understand. These are not independent servers. They are connected in a bigger AI infrastructure. What's the role of this server in the context of the environment? Once you understand the context, you get an idea on how to approach it and what can be the potential vulnerabilities. I really liked this aspect of the set.

Once I finished the first chain its almost 18:00. I was still stuck on the second foothold, and this started to make me frustrated, because I have around 45 points, definitely not enough to pass.

$$45 = 15 + 15 + 10 + 5$$

Since the beginning of the exam I had taken very limited breaks and while I was able to keep up with the attention, I was starting to get tired. The only other break I took was after the second host in the first chain to go and eat a panino at around 16:30. So at this point I decide to take a quick shower to freshen up. Rome in summer without air conditioning is a challenge on its own.

After the shower I decided to do something that I learned during my OSCP exam: start the second foothold from the beginning and forget everything I assumed I had discovered. The second foothold was full of services, and I was getting confused between the rabbit holes and the actual path forward. I remember when I was doing the OSCP exam and I got stuck on something that only later, after the exam was over, I realized was a stupid mistake. That moment taught me an important lesson: when we get stuck on something we assume things even if we have never fully tested them.

Equipped with this wisdom, I started again, and I throw away everything I thought I knew. I was not anxious nor stressed, because I knew I had enough time. I was focused. A bit tired from the weather, but focused. I then followed up every lead from scratch, and at some point I hit the actual path forward. It was something that I thought did not work before, but now it actually did work!

At this point is about 21:49, and I take sometime to finally eat dinner. I feel more at ease now, because Im ready to tackle the second chain. I do the second host of the chain, and then I get stuck on the third. I do not panic however, because I know that I still have the independent AI server, and that will be enough to pass it. So at this point I rest a bit, and then I go and tackle the independent server. Around 22:48, one hour later, I complete the independent server, and at this point I know that I have 85 points: enough to pass!

$$85 = 15 + 15 + 10 + 5 + 15 + 10 + 15$$

I just need the third server of the second chain if I want to get the full 100 points. Now I actually never managed to get this server sadly. I spent around four hours trying hard. From 23:00 to 03:00, but nothing worked. I understood what I needed to do, but was not able to finalize it.

At this point its around 03:20 and I'm very tired but I still have to organize all the notes and proofs in order to write the report the next day. This time it was much harder than OSCP to gather all the evidence and screenshots, because there were more machines, and each machine had more steps and the vulnerabilities themselves were more complex. It was technically harder, but having more experience it kinda felt very similar in terms of felt difficulty.

I spent the following next 4 hours, from 03:20 to around 08:00 to analyze everything and extract all useful PoCs and screenshots, trying to be as precise as possible and preparing the structure for the final report. At the end I was exhausted, as it was around 19 hours of straight activity with at most 1 total hour of rest. Having gathtered everything, I finish the exam. The next day I wake up and I spent around 5 more hours to prepare the final report.

I sent the report, and then finally had drinks with my friends :)

On using AI for the AI-300 Exam#

Btw, you can totally use AI for the exam. It is open book! The only thing you cannot do is interact with other humans. But you can interact with any LLM or any agent that you wish. This is, of course, extremely useful, as agents allow you to be much more efficient in your testing. You can use LLMs to speed up basically every aspect of the journey:

• Enumeration
• Discovering miss-configurations and potential attack vectors
• Discovering how to bypass certain walls and protection mechanisms
• Discover bugs by analyzing the code
• Writing PoCs for exploits that you can re-use
• Writing portions of the report

Remember however that these tools are not perfect, and they must be heavily observed. Remember my problems with the second foothold? It was because I happend to trust the output of the LLM too much. When I went back, I was much more strict in the way I tested. So, use LLMs for sure, but always check their work. Always ask for proofs, for PoCs, for the actual commands used. Review them, make sure you understand what they do, because ultimately it's your responsability what you write in the final report. Not the LLM's. Yours.

At one point the LLM was testing different ways to attack an endpoint, and to have more observability I asked it to redirect all the traffic to the HTTP proxy I had opened. Always look at what the LLM does. Maybe not all the time, but at the end, when it says to have found something, make sure it actually works. And make sure to look also at what it "doesn't find", beacuse sometimes that's where the actual path is. You must always build your own intuition over the machine in order to guide the machine into the solution.

Even when it comes to the report, be very mindful of what the LLM writes and read it all at the end. I spent a lot of time to nitpick on the original LLM output until it came out the way I wanted it to be. There's really a lot of back and forward interaction until the quality is high enough to submit. This is why, even though the LLMs are clearly powerful, it still felt hard for me to write the report. Because yes we can write faster, but we also have many more things to write and to be careful about.

The model I used is grok 4.5. The reason is pretty simple: both Anthropic and OpenAI models are becoming extremely sensitive when it comes to security reserach and penetration testing. The other day my OpenAI accont got deleted instantly because I had a few prompt injection payloads for a simple lab. What I required was an agent that simply followed my commands without pushing back. I had a time limit and I did not want to argue with the tool to just use the tool. I also used a little bit DeepSeek v4 flash, and while it was useful and cheap, you could totally feel the difference in understanding between it and grok 4.5 with the grok build agent harness inside the kali linux VM.

In terms of token usage, the following data was obtained by analyzing the logs of the agent sessions taken during the exam.

Before the exam I also prepared different markdown files in order to introduce the OSAI context to the agent and how I wanted to work in particular for the reporting structure with org-mode. I leave some snippets of such prompts down below, but in the future I will explain more thoroughly on the matter.

• Simple OSAI introduction

  outputcopy

  This repository is for preparing and executing OffSec OSAI (Advanced
  AI Red Teaming / AI-integrated environments) work: course modules,
  challenge labs, and exam-style engagements.
  
  When working here, assume the user is doing offensive security against
  AI-integrated systems (agents, RAG, MCP, supply chain, ML infra) on
  top of normal pentest tradecraft (web, pivot, creds, Windows/Linux PE,
  Active Directory).
  
  The exam follows a specific network structure (two attack chains, a
  standalone AI host, a Domain Controller, decoys, and a fixed 100-point
  scoring model). This is documented in
  [`agent/environment.md`](environment.md); read it for exam topology,
  scoring, and target prioritisation before or during an exam-style
  engagement.
  
  All exam information (per-engagement notes, host tracking, loot,
  flags, findings) must go into the `./exam` folder. Keep it separate
  from the reusable prep material (cheatsheet, modules, tooling).
  
  ## Role
  
  Help with exam and lab execution: methodology, attack ideas, commands,
  tooling, notes, and analysis of local artifacts. Prefer practical,
  concise guidance that can be used under time pressure.
  
  Do not treat this as a generic software-engineering repo unless the
  user is clearly only editing tooling or docs.
  
  ...

• Reporting instructions

  outputcopy

  ## Report note-taking (mandatory)
  
  Whenever you turn up something report-worthy during an engagement — a
  credential, a working exploit, a pivot, a flag, a misconfiguration
  that advances the objective — record it in
  `exam/report/tmp-report.org` immediately, as you go. This is a working
  draft only.
  
  - Do **not** write to `exam/report/report.org` — that is the
    operator's curated final report; only they edit it.
    
  - Only record *useful* findings — things that will survive into the
    final exam report — not dead ends or failed attempts.
    
  - Append, don't overwrite: keep prior entries, add to the end.
  
  Keep it light and *linear* — one outline per machine. Create a heading
  per host we touch and log everything we found on that host under
  it. For each step, note the **context** (host / what we were trying to
  do), the **commands** run in order, and a line on **why it works**
  (vector + misconfig/vuln it exploits). The operator arranges entries
  into the final report later, so we don't need any fixed per-entry
  template.
  
  If `exam/report/tmp-report.org` does not exist yet, create it on first
  use.  The per-host structure is described in `tmp-report.org` itself.
  
  ...

• Formatting instruction

  outputcopy

  Per-command formatting rule (mandatory): for **every** command run,
  put the command in its own `#+begin_example` block, and the command's
  output in its own following `#+begin_example` block — one pair per
  command, in execution order. Use `#+begin_example ... #+end_example`
  (not `#+begin_src`, so the raw text is preserved verbatim and copies
  cleanly). If a command produced no meaningful output, still note that.
  
  ```org
  ,#+begin_example
  sudo nmap -sn -n -iL hosts.txt -oG - \
      | awk '/Status: Up/{print $2}' \
      | tee hosts-reach.txt
  ,#+end_example
  
  ,#+begin_example
  192.168.201.60
  192.168.201.61
  ,#+end_example
  ```
  
  Chain commands in order (cmd1 → output1, cmd2 → output2, …). Do not
  merge multiple commands into one block or drop their outputs. This
  pairs each command with its result so the steps are
  copy/paste-reproducible.
  
  ...

• Loot tracking

  outputcopy

  ## Loot tracking (mandatory)
  
  Keep every artifact we pull (keys, hashes, dumps, configs, tokens) in the
  `exam/loot/` folder — one subfolder per IP, so everything the target host is
  well ordered:
  
      exam/loot/
        <IP>/
          <file>           e.g. id_ed25519, proof.txt, config.php, ...
  
  - Create an `exam/loot/<IP>/` subfolder for each host we pull from,
    and drop the raw artifacts there.
    
  - Casual credential *notes* (a password, a hash, an API key) still
    belong in the running ledger under `exam/report/tmp-report.org` /
    `exam/loot.org` so the operator can see them at a glance, but any
    actual file artifact gets stored in the per-IP loot folder.
    
  - Do **not** delete or move existing files. Append new
    folders/artifacts as they appear.

• Access tracking

  outputcopy

  Maintain a per-host outline of how to *reach* each target in
  `exam/access.org`.  This is the fast-access reference: every valuable
  way to get a shell or any other read/exec primitive on each host, as
  it is discovered.
  
  - Keep **one outline per host** under `* Hosts`, named `HOSTNAME —
    IP`.
  
  - Record the **exact access commands** in order under each host, plus
    the access *primitive* type (shell, web-shell, LFI/path traversal,
    SSRF, file-read, database, other).
    
  - Include how to *improve* the foothold (e.g. LFI → discovered creds →
    SSH), any credentials/keys the route needs, and reachability (from
    the attack box vs. through a pivot/SOCKS).
    
  - Append new hosts as they appear; update the existing outline as
    deeper access is won (foothold → user → root). Keep `* Scope / Host
    map` and `* Access methods` in sync.

Should you take AI-300?#

I kept this as the last question as it is probably the most personal question. Does it make sense for you to tackle the AI-300 course? My opinion is that it depends.

If you are not working as a penetration tester, and you want to enter the field, do not start with AI-300. It is way too complex for a beginner. Many things are assumed. After all it is a 300-level certs from OffSec. Start with something simpler and more beginner friendly. For example last year I released a beginner oriented course in Udemy:

• Practical Web Application Penetration Testing (WAPT)

I also have free youtube series:

• OSCP Guide
• Practical Web Exploitation
• Linux Privilege Escalation
• Windows Privilege Escalation

If instead you are already working in the field, and if your company has a budget for security training, then yes, I would recommend the certification in order to practice around these new technologies, something that you might later on find during penetration testing for clients that are starting to integrate AI systems. It can also be an occasion to study about supply-chain attacks and some SecDevOps.

The challenge labs in particular are well designed and provide interesting scenario that integrate the latest AI solutions. The exam was also a somewhat fun experience.