The Stellenbsoch University Division of Computer Science prizegiving took place on the 25th of April 2024. Praelexis sponsors the “Best Honours Project” award, which was awarded to Emile Ferreira for his performance in 2023. Emile chatted with Praelexis about his research, how his passion for computer science started, his view on Responsible AI and his plans for the future.
How did it start?
Can you share your journey into the world of computer science and machine learning? What sparked your interest in these fields, and how did you pursue your passion?
I learnt to type, on my dad's laptop, before I could write, and designed rudimentary computers, which I later built from recycled laptop components. My aptitude and passion for technology spurred me to learn to code in primary school. I remember the challenge and reward of building my first website. I began offering this service to friends, family and small businesses.
Late at night in the South African College High School boarding house, I worked remotely as a contracted software developer for Replit—then a startup, now a unicorn. I contributed to the Replit education platform, sharing my knowledge with a global audience through free, interactive coding courses. At the same time, I became interested in machine learning. After following some scikit-learn tutorials, I began building language models on my homemade computer. One of these models became Bestee; an offline, privacy-respecting virtual assistant for Android. Bestee was named the Most Innovative Solution at the MTN Business App of the Year awards.
I extended my knowledge through formal study at Stellenbosch University, with an undergraduate degree in mathematical sciences and an honours degree in computer science. I took modules in mathematical statistics and machine learning, and my honours research project focused on the transformer language model architecture. I readily shared this knowledge with my classmates, through regular workshops hosted by Debugged, and supervised 40 undergraduate students on a research project to understand the technology behind ChatGPT.
Tell us about your research
Winning a prize for academic excellence is a significant achievement. Could you walk us through the project or research that led to this recognition? What were the key insights or breakthroughs you uncovered?
Using formal language theory, I investigated the expressive power of transformers to determine what they're capable of and contribute towards their mechanistic interpretability. The research focused on context-free languages, which model hierarchical dependencies in both natural and programming languages (many of which are, themselves, context-free). I hope to publish my results soon.
How do you think your academic background and experiences have shaped your approach to problem-solving within computer science and machine learning?
Creativity is important to me. It heightens my problem-solving ability and sets me apart in the technology sector. It was fostered during Visual Arts classes in high school and extracurricular study at Putter School. The Allan Gray Orbis Foundation recognised my entrepreneurial drive and funded my university studies. They provide funding and training to develop high-impact, responsible entrepreneurs. Stellenbosch University has taught me the scientific method, perseverance and the importance of mathematics. With these tools, I am well-equipped to solve problems in computer science and machine learning.
In the rapidly evolving landscape of technology, what areas within computer science and machine learning do you find most intriguing or promising for future exploration?
I'm most interested in large language models. Using my knowledge of formal language theory, I would like to improve their performance on structured input and output.
Collaboration often plays a crucial role in academic success. Can you discuss any collaborations or mentorships that have influenced your growth as a computer scientist and machine learning enthusiast?
The guidance, freedom and ideas provided by my supervisor, Prof. Brink van der Merwe, were fundamental to the success of my research. I cannot thank the Stellenbosch University academic staff enough for their support. My supervisor also introduced me to the Formal Languages and Neural Networks (FLaNN) research group, where I learn from and collaborate with international academics at the forefront of neural language modelling and formal language theory.
Beyond academic achievements, what extracurricular activities or personal interests have complemented your journey in computer science and machine learning? How do they contribute to your overall perspective?
I'm a keen tennis player and runner. I played for Maties Tennis last year and completed the Two Oceans Half Marathon earlier this year. Sport helps me to maintain a work-life balance and avoid burnout during my studies.
As the field of machine learning continues to expand, ethical considerations become increasingly important. How do you approach ethical dilemmas or considerations in your research and practice?
Like Praelexis, I believe in the need for responsible artificial intelligence (AI). In my capacity, I build AI that's sensitive to the South African context and contribute to AI interpretability: understanding how AI works (and why it sometimes doesn't) and making that knowledge accessible. This builds trust and can help to uncover biases in the tools of tomorrow. I'm working to open-source my code and publish my research in an open-access journal.
Let's talk about the future
What advice would you offer to aspiring computer scientists and machine learning enthusiasts who aspire to achieve similar levels of academic excellence?
I suggest using free online resources to learn to code as early as possible. Be intentional in building and curating a personal brand, as it will help secure funding and employment. Attend workshops and relevant events to network with supervisors and companies. Use and contribute to free and open-source software whenever possible, to protect your privacy and grow a portfolio of work.
Innovation often stems from failure and learning from mistakes. Can you share any challenges or setbacks you've encountered along your academic journey and how you've overcome them?
In high school, I was placed in the bottom-set mathematics class, where we covered the curriculum painfully slowly and skipped challenging sections. This judgement spurred me to prioritise mathematics—hiring additional tutors and revising the work every evening. I noticed an improvement in my marks and soon developed a love of mathematics. Three years later, I graduated with a degree in mathematical sciences, cum laude.
Looking ahead, what are your aspirations and goals within the realm of computer science and machine learning? How do you envision making a meaningful impact in these fields?
I would like to make an impactful and ethical contribution to AI in Africa, through my research and entrepreneurial ventures. I hope to grow a successful software-as-a-service company that creates meaningful employment opportunities and provides human-centric solutions that are sensitive to the South African context. I plan to remain involved in academia, sharing my knowledge and inspiring the next generation of change-makers. I also hope to consult on legislation, at a national and global level, to help ensure responsible and accountable development of AI.
Praelexis would like to, once again, congratulate Emile Ferreira on his exceptional achievement.
Unlock the power of AI-driven solutions and make your business future-fit today! Contact us now to discover how cutting-edge large language models can elevate your company's performance:
