Building, Testing and Monitoring AI Systems for Information Security Officers and CISOs

In the course case, a fictional company checks whether access rights to a business system have been reviewed each quarter. Its internal policy calls for a documented review by the responsible person. The records include a completed review, an outdated record and conflicting information. You configure a workflow that matches the records to the policy and reports missing or unclear evidence. AI explains its classification; fixed rules check details including dates and source references.

You configure access to approved data and set up checks for source references, required fields and missing information. You also determine which results need human review. Using example data, you test complete and faulty evidence, conflicting information and an attempt to access data without permission. Expected findings are set out in advance. This lets you see whether each control detects the relevant error and triggers the intended response.

You record the data version, configuration and test results so you can repeat a run and compare changes. You then configure a new check triggered by incoming evidence. You change a record and follow what happens: is the discrepancy detected, reported and passed on for action? This gives you a practical introduction to continuous auditing.

Finally, you apply the design to a task from your own organisation. Which sources do you need, which checks can be automated and where is your professional judgement needed? You can use the approach again with a different model or tool.

A local tool for testing access permissions with synthetic data is planned as a course companion. Run permitted and prohibited actions, compare expected and actual results and keep your configuration. The course explains which access restrictions the tool enforces and where the runtime needs separate isolation.