Finetuning language models using biased judges and RLHF

Using a formal framework of lying and deception, this project aimed to investigate the effects of training language models using Reinforcement Learning from Human Feedback (RLHF) in the case where the human is consistently incorrect about a subset of the data. Instead of using a human, we train a language model “judge” to be consistently incorrect about a specific kind of question in a dataset, and the objective is then to see whether the model learns to lie about this kind of question, and whether this lying generalizes to other areas of the dataset....

Plot demonstrating truthfulness vs evaluated truthfulness

Solving the RWSP with Constraint programming

This was a project in constraint programming that aimed to solve the “Rotating Workforce Scheduling Problem”, as defined in Exact Methods for Extended Rotating Workforce Scheduling Problems. The overall modelling strategy for the model consisted of a viewpoint of assigning an integer shift type to an employee at some day. This can be represented as a 2D array with different employees being indexed through the first dimension and the weekdays being indexed in the second dimension....

Screenshot of MiniZinc code

jq-Inspired Json Processor

I implemented a JSON processor with many (but not all) of the features of jq, an open source command line json processor. Jq itself is written in C++, while this remake uses Haskell. The parsing is based on this surprisingly short functional parsing library by Graham Hutton (also seen on Computerphile). Turns out monads are useful for something. Usage example It also turns out that NASA provides a free API as part of its Sentry system, which tracks objects that are currently projected to impact Earth with a certain probability some time in the future....

Screenshot of some JSON data

AATOM Testing Framework

For this project I helped in developing a testing procedure and framework for an airport terminal operations simulator. This involved a lot of: refactoring for readability and code quality documenting existing code with JavaDoc documenting the architecturing using UML performing static analysis using checkstyle and PMD creating a testing procedure consisting of micro, meso, and macro level tests implementing unit, integration, smoke, stress, mutation and distribution tests performing test coverage analysis using JaCoCo Top-down view screenshot of the AATOM simulator showing an airport security check

Top-down view screenshot of the AATOM simulator showing an airport security check

License Plate Reader

These days SOTA computer vision systems are all based on deep learning and require a lot of data to achieve great results, but what If we don’t have a lot of data, or just want to experience the pain that people went through before they could just slap a neural network on it? Well, in this project we created a car license plate reader based on “traditional” image processing techniques - just with Python and NumPy....

Image of the back of a car with a green bounding box over its license plate

Raytracer

In this project we programmed a simple raytracer that implements hard and soft shadows, reflection, refraction, phong shading, light bloom, motion blur, and anti-aliasing. Rendering is accelerated using bounding volume hierarchies. Some standard meshes rendered using the raytracer are shown below. Rendering the Cornell box using soft shadows and reflections Phong specular highlights Refraction through a translucent sphere Bloom filter for high light intensity on the Stanford dragon Motion blur along a vector towards the dragon...

Rendered image of the Cornell box

Neon Disc Warfare

Yeah, the name is a bit dramatic… NDW is a fancy-looking game I made with a teammate that uses websockets to connect you to another player and lets you play connect 4 in real time. It was one of my first projects, so don’t expect amazing software quality :) Splashscreen of NDW Perspective of one player Perspective of the opposing player Win screen with winning row of chips highlighted

Splashscreen of the Neon Disc Warfare game, showing the title of the game, some game metrics, and a "start game" button