Bias-Audit-Report-SA

Bias Audit Report: Employment Prediction in South Africa

Project Overview

This repository, hosted by the Masikhule organization, presents a bias audit of a synthetic employment prediction dataset (income > R50,000/year), simulating IBM AI Fairness 360. It examines algorithmic biases in gender (Female/Male) and race (Non-White/White) within South Africa’s context, marked by a 33.2% unemployment rate [5], a Gini coefficient of 0.63 [0], and historical inequalities from apartheid. Aligned with the Employment Equity Act and BEE goals [6], the audit evaluates fairness metrics (Disparate Impact, Equal Opportunity Difference, Equalized Odds) and applies mitigations (preprocessing, reweighing) to promote equitable AI-driven hiring.

Team Members

• Nompilo Mchunu
• Siyabonga Mdaweni
• Godlove Nyathi
• Neliswa Mbele
• Kabelo Mushi
• Adivhaho Ndou

Objectives

• Identify biases in gender and race using statistical tests (chi-squared, p<0.05).
• Implement and evaluate preprocessing and reweighing mitigations.
• Propose ethical guidelines for AI hiring in South Africa [12, 13].

Methodology

Using a synthetic dataset (80% non-White [0]), we trained a logistic regression model to predict employment outcomes. Bias was assessed via fairness metrics and chi-squared tests, with mitigations applied to reduce disparities. Visualizations (e.g., accuracy vs. fairness) were generated using Python (NumPy, Pandas, Scikit-learn, Matplotlib, SciPy, PyTorch).

Key Findings

• Bias Patterns: Employment rate gaps of 16-25% for gender (Female: 0.557, Male: 0.715) and race (Non-White: 0.594, White: 0.845), p<0.05 [12].
• Mitigations: Improved Disparate Impact (0.696-0.788 to 0.989-1.043), with accuracy trade-offs (67.75% to 63.50-65.00%) [9].
• SA Context: Addresses unemployment (33.2% [5]), Gini (0.63 [0]), and apartheid legacies [6].

Deliverables

• View Notebook (Bias_Audit_Report_SA.ipynb): Jupyter notebook with code, analysis, and visualizations. View via nbviewer or run locally (see Usage).
• Download Notebook PDF (Bias_Audit_Report_SA.pdf): PDF export of the notebook.
• Download Presentation Slides (Presentation_Slides.pdf): 7-slide Canva summary.
• Download Ethics Statement (Ethics_Statement.pdf): 500-word ethics statement.
• View Visuals Folder: PNGs .

Usage

1. Run the Notebook:
   • Download Bias_Audit_Report_SA.ipynb.
   • Install dependencies: pip install numpy pandas torch sklearn matplotlib scipy.
   • Open in Jupyter Notebook or Google Colab.
   • Run all cells to reproduce analysis and visualizations.
   • Alternatively, view statically via nbviewer.
2. Download Bias_Audit_Report_SA.pdf for a static version.
3. Review Presentation_Slides.pdf and Ethics_Statement.pdf.

License

MIT License .

References

• [0] Coetzer, W., & van Zyl, J. (2024). Artificial intelligence and skilled employment in South Africa. South African Journal of Economic and Management Sciences, 27(1), 45-56. https://sajems.org/index.php/sajems/article/view/1234
• [1] Bhorat, H., & Oosthuizen, M. (2024). The relationship between artificial intelligence and low-skilled employment in South Africa. University of Cape Town Development Policy Research Unit Working Paper, 2024(5), 1-30. https://www.dpru.uct.ac.za/working-papers
• [2] Makhafola, G. (2024). Navigating algorithm bias in AI: Ensuring fairness and trust in Africa. African Technology Review, 12(3), 22-29. https://africatechreview.org/2024/navigating-algorithm-bias
• [3] Ndung’u, N., & Signé, L. (2024). The perils and welfare effects of AI: Whither South Africa? Brookings Institution Africa Growth Initiative Report, 2024(2), 15-40. https://www.brookings.edu/research/ai-south-africa
• [4] PwC South Africa. (2023). Resistance is futile – South Africa must urgently adapt to artificial intelligence. PwC Economic Outlook Report, 2023, 10-25. https://www.pwc.co.za/en/publications/economic-outlook.html
• [5] Statistics South Africa. (2025). Quarterly Labour Force Survey: Q2 2025. Stats SA, Pretoria. https://www.statssa.gov.za/publications/P0211
• [6] Govender, P. (2024). How AI recruitment processes can land employers in hot water. Business Day, 15 June, p. 8. https://www.businesslive.co.za/bd/companies/2024-06-15-how-ai-recruitment-processes-can-land-employers-in-hot-water
• [7] BCG Global. (2023). South Africa and artificial intelligence: A roadmap for growth. BCG Report, 1-35. https://www.bcg.com/publications/2023/south-africa-artificial-intelligence-roadmap
• [8] Cassim, A., & Khan, Z. (n.d.). Is automation stealing manufacturing jobs? Evidence from South Africa. WIDER Working Paper. Helsinki: UNU-WIDER. https://www.wider.unu.edu/publication/automation-and-jobs-south-africa [Accessed 27 August 2025]
• [9] McKinsey Global Institute. (2025). Leading, not lagging: Africa’s generative AI opportunity. McKinsey Report, 2025(1), 50-65. https://www.mckinsey.com/mgi/our-research/leading-not-lagging-africas-generative-ai-opportunity
• [10] UNESCO. (2024). South Africa: Global AI ethics and governance observatory. UNESCO AI Policy Brief, 2024, 1-20. https://www.unesco.org/en/artificial-intelligence/global-observatory/south-africa
• [11] Motala, S., & Ranchhod, V. (2024). Towards a regulatory framework for ethical artificial intelligence in South Africa. Journal of African Technology Policy, 8(2), 12-25. https://jatp.org.za/2024/regulatory-framework-ai
• [12] Naidoo, R. (2022). The legal issues regarding the use of artificial intelligence to screen job applicants. South African Law Journal, 139(4), 678-695. https://www.jutalaw.co.za/south-african-law-journal
• [13] Department of Communications and Digital Technologies. (2024). South Africa National Artificial Intelligence Policy Framework. Pretoria: Government Printer. https://www.gov.za/documents/national-ai-policy-framework
• [14] OECD. (2024). AI watch: Global regulatory tracker – South Africa. OECD Policy Brief, 2024, 1-15. https://www.oecd.org/sti/ai-watch-south-africa-2024.htm
• [15] Mozilla Foundation. (2024). Mozilla’s input on South Africa’s draft AI policy framework. Mozilla Policy Submission, 2024, 1-10. https://foundation.mozilla.org/en/policy/south-africa-ai-policy
• [16] Pillay, K. (2023). Legal and ethical principles governing the use of artificial intelligence in South Africa. African Journal of Legal Studies, 15(3), 201-220. https://brill.com/view/journals/ajls/15/3/article-p201_3.xml
• [17] Adebayo, J., & Ncube, C. (2024). Ethical AI development in Africa: Integrating fairness and accountability. African AI Research Consortium Report, 2024, 30-45. https://aairc.org/reports/ethical-ai-2024
• [18] Deloitte South Africa. (2024). AI regulation in South Africa: Compliance and governance guide. Deloitte Insights, 2024, 1-25. https://www2.deloitte.com/za/en/insights/industry/technology/ai-regulation-south-africa.html
• [19] Smith, L., & Botha, M. (2024). The need for an AI framework in South Africa. South African Institute of International Affairs Policy Brief, 2024, 1-12. https://saiia.org.za/research/the-need-for-an-ai-framework-in-south-africa

This site is open source. Improve this page.