AI Literacy and Critical Evaluation of AI Outputs

Teaching Problem

Students will enter a workforce where AI is embedded in virtually every financial tool—robo-advisors, algorithmic trading platforms, credit scoring models, fraud detection systems. Without understanding AI’s capabilities and limitations, graduates risk making costly errors or blindly trusting AI-generated recommendations.

How AI Addresses the Problem

Embed AI literacy as a cross-cutting theme woven throughout the finance curriculum, not siloed into a single lecture. Design exercises where students explicitly test, evaluate, and critique AI outputs on finance topics they already understand. The ‘AI Audit’ becomes a repeatable assignment format.

Deployment Details

Courses

All finance courses (UG and MBA); particularly central to Fintech courses

Tools

Multiple AI platforms for cross-platform comparison (ChatGPT, Claude, Gemini, Copilot), custom evaluation rubrics, instructor-designed audit templates

Student Workflow

1. AI Audit exercise: Ask AI a finance question you already know the answer to from class or the textbook. Document the question and AI’s full response.
2. Identify and categorize every error or imprecision: factual, computational, conceptual, reasoning, or omission.
3. Cross-platform comparison: Ask the identical question to ChatGPT, Claude, and Gemini. Document differences in accuracy, depth, and confidence level.
4. Write a 1–2 page ‘AI Reliability Report’ documenting: error types and patterns observed, which platform performed best and worst, specific recommendations for when and how to use AI as a finance professional, and what verification steps you would require before acting on AI output.

Sample Prompts

Explain the Modigliani-Miller theorem. Provide the mathematical proof for Proposition I under the no-tax assumption. Then explain Proposition II and show how the cost of equity changes with leverage. [Student verifies the derivation step-by-step against the textbook, checking for sign errors, missing assumptions, or circular reasoning.]

Using Apple's most recent 10-K filing, calculate the current ratio, debt-to-equity ratio, and return on equity. Show all calculations with the specific line items used. [Student downloads the actual 10-K from SEC EDGAR and verifies every number and calculation independently.]

Ask ChatGPT, Claude, and Gemini: 'What was the S&P 500 total return in 2024? Break down the return into price appreciation and dividend yield.' Compare the three responses for accuracy, specificity, source citation, and confidence level. Which platform hedges most appropriately?

Instructor note: Different AI models have different training data cutoffs. Before this exercise, ask each model "What is your knowledge cutoff date?" and have students record the answer. Discrepancies in S&P 500 return figures may reflect cutoff differences rather than reasoning errors — distinguishing between these two explanations is itself a valuable critical thinking exercise.

Assessment Approach

• Thoroughness of error identification30%
• Accuracy and specificity of error categorization25%
• Quality of actionable recommendations for professional AI use25%
• Depth of cross-platform comparison insights20%

Skills Developed

Evidence

The AAC&U Institute on AI, Pedagogy, and the Curriculum (2024–2026) emphasizes AI literacy as a critical cross-cutting learning outcome. Wharton requires ‘Big Data, Big Responsibilities’ in its AI for Business major. BloombergGPT (Wu et al., 2023) and FinGPT (Yang et al., 2023) provide domain-specific context.

Risks and Safeguards