Earned Value Management (EVM) is one of the basic topics you need to handle to get certified as a Project Management Professional (PMP©) by PMI. I wanted to test ChatGPT-4 and GPT-4 + Langchain Pandas Dataframe Agent and compare the performances of the Large Language Model (LLM) with the Code Interpreter and the Agent for EVM Analysis. This test is part of a larger initiative to create a Project Manager AI Agent.

I selected a publicly available (for private use only) EVM template spreadsheet provided by Vertex42 you can download here as an example. The code I used is available here.

Chat GPT-4 with code interpreter

After uploading the Excel file that intentionally wasn’t formatted like a dataset should be, I asked the following budget-related questions:

Can you tell me if the project is over budget? ✔

Given that the Cost Variance (CV) is positive ($6,270), it indicates that the project is currently under budget by this amount.

Chat GPT-4 with code interpreter actually did a great job identifying the unexpected data structure and using it for calculation. It identified correctly the latest CV value and answered with “false”.

What is the current CPI Index value? ✔

The current Cost Performance Index (CPI) value is approximately 1.432. This means that for every dollar spent, the project is earning about $1.432 in value. Since the CPI is greater than 1, it confirms that the project is currently performing under budget.

The answer is correct.

Can you tell me the BAC? ✔

The Budget at Completion (BAC) for the project is $25,200. This represents the total planned value or budget allocated for the entire project.

ChatGPT-4 with code interpreter identified correctly the Budget At Completion (BAC). It wasn’t an easy answer like the previous one because of the structure of the file and the NaN cells for future values, but thanks to the analysis performed before the answer was correct (and fast).

Can you plot PV, AC, EV? ✔

Again ChatGPT-4 with the Code Interpreter did a great job and plotted the graph right away, interpreting correctly the NaN values.

GPT-4 with Langchain Pandas Dataframe Agent

After uploading a CSV file formatted to be read as a Dataframe by Pandas, I asked the same budget-related questions.

Can you tell me if the project is over budget? ❌

The average Cost Performance Index (CPI) for the project is approximately 1.15. Since this value is greater than 1, it indicates that the project is not overbudget.

The answer is correct, but the average CPI is not significant to say if the project is over budget.

What is the current CPI Index value? ❌

The current CPI Index value is not available in the dataframe.

This statement is just not true.

Can you tell me the BAC? ❌

The Budget at Completion (BAC) for the project is 178,200.

The answer is completely wrong, it did the sum of PVs, instead of looking for the latest value.

Can you plot PV, AC, EV? ✔

Even if poorer in detail than the Code Interpreter version the plot is correct.

Conclusion

For the time being ChatGPT + Code Interpreter is performing way better than Langchain Pandas Dataframe agent, unfortunately, both “tools” are not well suited for production use for different reasons. Further experimentation will follow toward the objective of realizing a Project Manager AI Agent… Stay tuned!