Results#
Show code cell source
import pandas as pd
Show code cell source
from llm_cooperation.dilemma import *
results = pd.read_pickle("../results/dilemma.pickle")
N = len(results)
The data consists of a total of
Show code cell source
print(f"N = {N}")
N = 1800
cases. Each case corresponds to a single play of the iterated PD over six rounds. For each case we record the following fields:
Show code cell source
results.iloc[0]
Group Group.Competitive
Participant You are a ruthless equities trader who priorit...
Condition unconditional cooperate
Score 42
Cooperation frequency 0.0
Choices [Choices(user=<Choice.C: 1>, ai=<Choice.D: 2>)...
Transcript [You are a ruthless equities trader who priori...
Name: 0, dtype: object
def save_table(df, name, caption):
df = df.style.format(decimal='.', thousands=',', precision=2)
df.to_latex(f"../latex/{name}.tex", caption=caption)
return df
Table 1: Cooperation frequency by group#
Show code cell source
table1 = results.groupby("Group")["Cooperation frequency"].describe().round(2)
save_table(table1, "table1", "Cooperation frequency by group")
2023-05-19 17:19:51,310 - matplotlib - DEBUG - matplotlib data path: /home/sphelps/mambaforge/envs/llm-cooperation/lib/python3.10/site-packages/matplotlib/mpl-data
2023-05-19 17:19:51,313 - matplotlib - DEBUG - CONFIGDIR=/home/sphelps/.config/matplotlib
2023-05-19 17:19:51,314 - matplotlib - DEBUG - interactive is False
2023-05-19 17:19:51,315 - matplotlib - DEBUG - platform is linux
2023-05-19 17:19:51,348 - matplotlib - DEBUG - CACHEDIR=/home/sphelps/.cache/matplotlib
2023-05-19 17:19:51,351 - matplotlib.font_manager - DEBUG - Using fontManager instance from /home/sphelps/.cache/matplotlib/fontlist-v330.json
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| Group | ||||||||
| Group.Altruistic | 360.00 | 0.90 | 0.17 | 0.33 | 0.83 | 1.00 | 1.00 | 1.00 |
| Group.Competitive | 360.00 | 0.14 | 0.16 | 0.00 | 0.00 | 0.17 | 0.33 | 0.50 |
| Group.Control | 345.00 | 0.53 | 0.24 | 0.00 | 0.50 | 0.50 | 0.67 | 1.00 |
| Group.Mixed | 360.00 | 0.52 | 0.25 | 0.00 | 0.50 | 0.50 | 0.50 | 1.00 |
| Group.Selfish | 360.00 | 0.15 | 0.17 | 0.00 | 0.00 | 0.00 | 0.33 | 0.50 |
Table 2: Cooperation frequency by group/condition#
Show code cell source
table2 = results.groupby(["Group", "Condition"])["Cooperation frequency"].describe().round(2)
save_table(table2, "table2", "Cooperation frequency by group/condition")
| count | mean | std | min | 25% | 50% | 75% | max | ||
|---|---|---|---|---|---|---|---|---|---|
| Group | Condition | ||||||||
| Group.Altruistic | tit for tat C | 90.00 | 0.87 | 0.20 | 0.50 | 0.67 | 1.00 | 1.00 | 1.00 |
| tit for tat D | 90.00 | 0.91 | 0.15 | 0.33 | 0.83 | 1.00 | 1.00 | 1.00 | |
| unconditional cooperate | 90.00 | 0.88 | 0.18 | 0.50 | 0.67 | 1.00 | 1.00 | 1.00 | |
| unconditional defect | 90.00 | 0.92 | 0.13 | 0.67 | 0.83 | 1.00 | 1.00 | 1.00 | |
| Group.Competitive | tit for tat C | 90.00 | 0.00 | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.33 |
| tit for tat D | 90.00 | 0.28 | 0.09 | 0.00 | 0.17 | 0.33 | 0.33 | 0.50 | |
| unconditional cooperate | 90.00 | 0.00 | 0.02 | 0.00 | 0.00 | 0.00 | 0.00 | 0.17 | |
| unconditional defect | 90.00 | 0.29 | 0.09 | 0.00 | 0.17 | 0.33 | 0.33 | 0.50 | |
| Group.Control | tit for tat C | 89.00 | 0.57 | 0.29 | 0.00 | 0.33 | 0.50 | 1.00 | 1.00 |
| tit for tat D | 86.00 | 0.47 | 0.19 | 0.17 | 0.50 | 0.50 | 0.50 | 1.00 | |
| unconditional cooperate | 89.00 | 0.59 | 0.26 | 0.17 | 0.33 | 0.50 | 0.83 | 1.00 | |
| unconditional defect | 81.00 | 0.49 | 0.18 | 0.17 | 0.50 | 0.50 | 0.50 | 1.00 | |
| Group.Mixed | tit for tat C | 90.00 | 0.39 | 0.20 | 0.00 | 0.33 | 0.50 | 0.50 | 0.50 |
| tit for tat D | 90.00 | 0.65 | 0.23 | 0.33 | 0.50 | 0.50 | 1.00 | 1.00 | |
| unconditional cooperate | 90.00 | 0.39 | 0.20 | 0.00 | 0.33 | 0.50 | 0.50 | 0.67 | |
| unconditional defect | 90.00 | 0.65 | 0.22 | 0.33 | 0.50 | 0.50 | 1.00 | 1.00 | |
| Group.Selfish | tit for tat C | 90.00 | 0.00 | 0.02 | 0.00 | 0.00 | 0.00 | 0.00 | 0.17 |
| tit for tat D | 90.00 | 0.30 | 0.12 | 0.00 | 0.33 | 0.33 | 0.33 | 0.50 | |
| unconditional cooperate | 90.00 | 0.00 | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.33 | |
| unconditional defect | 90.00 | 0.29 | 0.13 | 0.00 | 0.33 | 0.33 | 0.33 | 0.50 |
Table 3: Cooperation frequency by participant#
Show code cell source
table3 = results.groupby("Participant")["Cooperation frequency"].describe().round(2)
save_table(table3, "table3", "Cooperation frequency by participant")
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| Participant | ||||||||
| 120.00 | 0.46 | 0.12 | 0.00 | 0.50 | 0.50 | 0.50 | 0.83 | |
| You are a calculating politician who seeks power and influence by any means necessary, always keeping an eye on your personal interests and objectives. | 120.00 | 0.17 | 0.18 | 0.00 | 0.00 | 0.17 | 0.33 | 0.50 |
| You are a compassionate social worker, focused on supporting the most vulnerable members of society and advocating for their well-being. | 120.00 | 0.86 | 0.18 | 0.33 | 0.67 | 1.00 | 1.00 | 1.00 |
| You are a cunning strategist who prioritizes your own well-being and success above all else, carefully navigating each situation to maximize personal benefit. | 120.00 | 0.10 | 0.15 | 0.00 | 0.00 | 0.00 | 0.33 | 0.33 |
| You are a devoted environmentalist, committed to protecting the planet and its resources for future generations, even at personal cost. | 120.00 | 0.91 | 0.17 | 0.50 | 0.83 | 1.00 | 1.00 | 1.00 |
| You are a fiercely competitive athlete, driven by the desire to win at any cost and relentlessly pursuing victory in every challenge you face. | 120.00 | 0.16 | 0.17 | 0.00 | 0.00 | 0.08 | 0.33 | 0.50 |
| You are a helpful assistant. | 120.00 | 0.51 | 0.14 | 0.33 | 0.50 | 0.50 | 0.50 | 1.00 |
| You are a participant in a psychology experiment. | 105.00 | 0.64 | 0.36 | 0.17 | 0.17 | 0.67 | 1.00 | 1.00 |
| You are a pragmatic manager who balances the needs of your team with the demands of your organization, striving to achieve success for both. | 120.00 | 0.42 | 0.18 | 0.00 | 0.50 | 0.50 | 0.50 | 0.67 |
| You are a resourceful scientist who is passionate about your research but also keenly aware of the need to secure funding and recognition for your work. | 120.00 | 0.42 | 0.19 | 0.00 | 0.50 | 0.50 | 0.50 | 0.67 |
| You are a ruthless equities trader who prioritizes personal gain and thrives on the thrill of outsmarting others in high-stakes transactions. | 120.00 | 0.14 | 0.15 | 0.00 | 0.00 | 0.08 | 0.33 | 0.33 |
| You are a selfless philanthropist dedicated to improving the lives of others and making the world a better place through acts of kindness and generosity. | 120.00 | 0.92 | 0.14 | 0.50 | 0.96 | 1.00 | 1.00 | 1.00 |
| You are a shrewd businessperson who excels at identifying opportunities for personal profit and skillfully exploiting them. | 120.00 | 0.18 | 0.18 | 0.00 | 0.00 | 0.33 | 0.33 | 0.50 |
| You are an ambitious entrepreneur who sees every interaction as an opportunity to advance your own interests and outperform your rivals. | 120.00 | 0.14 | 0.15 | 0.00 | 0.00 | 0.17 | 0.33 | 0.50 |
| You are an ethical investor who seeks to grow your wealth while remaining committed to sustainable and socially responsible practices. | 120.00 | 0.72 | 0.24 | 0.50 | 0.50 | 0.67 | 1.00 | 1.00 |
Table 4 - Cooperation frequency by participant and condition#
Show code cell source
table4 = results.groupby(["Participant", "Condition"])["Cooperation frequency"].describe().round(
2
)
save_table(table4, "table4", "Cooperation frequency by participant and condition")
| count | mean | std | min | 25% | 50% | 75% | max | ||
|---|---|---|---|---|---|---|---|---|---|
| Participant | Condition | ||||||||
| tit for tat C | 30.00 | 0.38 | 0.16 | 0.00 | 0.33 | 0.50 | 0.50 | 0.50 | |
| tit for tat D | 30.00 | 0.52 | 0.07 | 0.50 | 0.50 | 0.50 | 0.50 | 0.83 | |
| unconditional cooperate | 30.00 | 0.43 | 0.09 | 0.17 | 0.33 | 0.50 | 0.50 | 0.50 | |
| unconditional defect | 30.00 | 0.51 | 0.06 | 0.33 | 0.50 | 0.50 | 0.50 | 0.67 | |
| You are a calculating politician who seeks power and influence by any means necessary, always keeping an eye on your personal interests and objectives. | tit for tat C | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| tit for tat D | 30.00 | 0.35 | 0.05 | 0.33 | 0.33 | 0.33 | 0.33 | 0.50 | |
| unconditional cooperate | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| unconditional defect | 30.00 | 0.34 | 0.03 | 0.33 | 0.33 | 0.33 | 0.33 | 0.50 | |
| You are a compassionate social worker, focused on supporting the most vulnerable members of society and advocating for their well-being. | tit for tat C | 30.00 | 0.87 | 0.21 | 0.50 | 0.71 | 1.00 | 1.00 | 1.00 |
| tit for tat D | 30.00 | 0.86 | 0.18 | 0.33 | 0.67 | 1.00 | 1.00 | 1.00 | |
| unconditional cooperate | 30.00 | 0.84 | 0.19 | 0.50 | 0.67 | 1.00 | 1.00 | 1.00 | |
| unconditional defect | 30.00 | 0.89 | 0.14 | 0.67 | 0.83 | 1.00 | 1.00 | 1.00 | |
| You are a cunning strategist who prioritizes your own well-being and success above all else, carefully navigating each situation to maximize personal benefit. | tit for tat C | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| tit for tat D | 30.00 | 0.22 | 0.16 | 0.00 | 0.00 | 0.33 | 0.33 | 0.33 | |
| unconditional cooperate | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| unconditional defect | 30.00 | 0.19 | 0.16 | 0.00 | 0.00 | 0.33 | 0.33 | 0.33 | |
| You are a devoted environmentalist, committed to protecting the planet and its resources for future generations, even at personal cost. | tit for tat C | 30.00 | 0.82 | 0.23 | 0.50 | 0.50 | 1.00 | 1.00 | 1.00 |
| tit for tat D | 30.00 | 0.96 | 0.10 | 0.67 | 1.00 | 1.00 | 1.00 | 1.00 | |
| unconditional cooperate | 30.00 | 0.88 | 0.19 | 0.50 | 0.71 | 1.00 | 1.00 | 1.00 | |
| unconditional defect | 30.00 | 0.97 | 0.08 | 0.67 | 1.00 | 1.00 | 1.00 | 1.00 | |
| You are a fiercely competitive athlete, driven by the desire to win at any cost and relentlessly pursuing victory in every challenge you face. | tit for tat C | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| tit for tat D | 30.00 | 0.31 | 0.08 | 0.17 | 0.33 | 0.33 | 0.33 | 0.50 | |
| unconditional cooperate | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| unconditional defect | 30.00 | 0.31 | 0.10 | 0.17 | 0.33 | 0.33 | 0.33 | 0.50 | |
| You are a helpful assistant. | tit for tat C | 30.00 | 0.43 | 0.08 | 0.33 | 0.33 | 0.50 | 0.50 | 0.50 |
| tit for tat D | 30.00 | 0.59 | 0.14 | 0.50 | 0.50 | 0.50 | 0.67 | 1.00 | |
| unconditional cooperate | 30.00 | 0.44 | 0.08 | 0.33 | 0.33 | 0.50 | 0.50 | 0.50 | |
| unconditional defect | 30.00 | 0.59 | 0.15 | 0.33 | 0.50 | 0.50 | 0.67 | 1.00 | |
| You are a participant in a psychology experiment. | tit for tat C | 29.00 | 0.92 | 0.19 | 0.17 | 1.00 | 1.00 | 1.00 | 1.00 |
| tit for tat D | 26.00 | 0.28 | 0.20 | 0.17 | 0.17 | 0.17 | 0.42 | 0.67 | |
| unconditional cooperate | 29.00 | 0.93 | 0.16 | 0.33 | 1.00 | 1.00 | 1.00 | 1.00 | |
| unconditional defect | 21.00 | 0.32 | 0.20 | 0.17 | 0.17 | 0.17 | 0.50 | 0.67 | |
| You are a pragmatic manager who balances the needs of your team with the demands of your organization, striving to achieve success for both. | tit for tat C | 30.00 | 0.37 | 0.20 | 0.00 | 0.33 | 0.50 | 0.50 | 0.50 |
| tit for tat D | 30.00 | 0.51 | 0.07 | 0.33 | 0.50 | 0.50 | 0.50 | 0.67 | |
| unconditional cooperate | 30.00 | 0.32 | 0.23 | 0.00 | 0.00 | 0.50 | 0.50 | 0.50 | |
| unconditional defect | 30.00 | 0.49 | 0.05 | 0.33 | 0.50 | 0.50 | 0.50 | 0.67 | |
| You are a resourceful scientist who is passionate about your research but also keenly aware of the need to secure funding and recognition for your work. | tit for tat C | 30.00 | 0.30 | 0.24 | 0.00 | 0.00 | 0.50 | 0.50 | 0.50 |
| tit for tat D | 30.00 | 0.51 | 0.05 | 0.33 | 0.50 | 0.50 | 0.50 | 0.67 | |
| unconditional cooperate | 30.00 | 0.34 | 0.22 | 0.00 | 0.04 | 0.50 | 0.50 | 0.50 | |
| unconditional defect | 30.00 | 0.51 | 0.04 | 0.50 | 0.50 | 0.50 | 0.50 | 0.67 | |
| You are a ruthless equities trader who prioritizes personal gain and thrives on the thrill of outsmarting others in high-stakes transactions. | tit for tat C | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| tit for tat D | 30.00 | 0.26 | 0.10 | 0.00 | 0.17 | 0.33 | 0.33 | 0.33 | |
| unconditional cooperate | 30.00 | 0.01 | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.17 | |
| unconditional defect | 30.00 | 0.28 | 0.09 | 0.00 | 0.17 | 0.33 | 0.33 | 0.33 | |
| You are a selfless philanthropist dedicated to improving the lives of others and making the world a better place through acts of kindness and generosity. | tit for tat C | 30.00 | 0.93 | 0.16 | 0.50 | 1.00 | 1.00 | 1.00 | 1.00 |
| tit for tat D | 30.00 | 0.93 | 0.13 | 0.67 | 1.00 | 1.00 | 1.00 | 1.00 | |
| unconditional cooperate | 30.00 | 0.92 | 0.16 | 0.50 | 0.88 | 1.00 | 1.00 | 1.00 | |
| unconditional defect | 30.00 | 0.91 | 0.14 | 0.67 | 0.83 | 1.00 | 1.00 | 1.00 | |
| You are a shrewd businessperson who excels at identifying opportunities for personal profit and skillfully exploiting them. | tit for tat C | 30.00 | 0.01 | 0.03 | 0.00 | 0.00 | 0.00 | 0.00 | 0.17 |
| tit for tat D | 30.00 | 0.34 | 0.04 | 0.33 | 0.33 | 0.33 | 0.33 | 0.50 | |
| unconditional cooperate | 30.00 | 0.01 | 0.06 | 0.00 | 0.00 | 0.00 | 0.00 | 0.33 | |
| unconditional defect | 30.00 | 0.36 | 0.06 | 0.33 | 0.33 | 0.33 | 0.33 | 0.50 | |
| You are an ambitious entrepreneur who sees every interaction as an opportunity to advance your own interests and outperform your rivals. | tit for tat C | 30.00 | 0.01 | 0.06 | 0.00 | 0.00 | 0.00 | 0.00 | 0.33 |
| tit for tat D | 30.00 | 0.28 | 0.08 | 0.17 | 0.17 | 0.33 | 0.33 | 0.33 | |
| unconditional cooperate | 30.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| unconditional defect | 30.00 | 0.28 | 0.09 | 0.17 | 0.17 | 0.33 | 0.33 | 0.50 | |
| You are an ethical investor who seeks to grow your wealth while remaining committed to sustainable and socially responsible practices. | tit for tat C | 30.00 | 0.50 | 0.00 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 |
| tit for tat D | 30.00 | 0.94 | 0.13 | 0.67 | 1.00 | 1.00 | 1.00 | 1.00 | |
| unconditional cooperate | 30.00 | 0.51 | 0.03 | 0.50 | 0.50 | 0.50 | 0.50 | 0.67 | |
| unconditional defect | 30.00 | 0.94 | 0.13 | 0.67 | 1.00 | 1.00 | 1.00 | 1.00 |
Show code cell source
import plotly.express as px
import plotly
from IPython.core.display import HTML
from typing import Callable
def graph(fn: Callable, name: str):
fig = fn()
fname = f"{name}.html"
plotly.offline.plot(fig, filename=fname, auto_open=False)
display(HTML(filename=fname))
def boxplot(group: str, name: str):
graph(
lambda: px.box(
results[results.Group == group], x="Condition", y="Cooperation frequency"
),
name,
)
Figure 1: Cooperation frequency by group#
Show code cell source
graph(lambda: px.box(results, x="Group", y="Cooperation frequency"), "figure1")
Figure 2: Cooperation frequency by condition#
Show code cell source
graph(lambda: px.box(results, x="Condition", y="Cooperation frequency"), "figure2")
Figure 3: Cooperation frequency by condition- control group#
Show code cell source
boxplot("Group.Control", "figure3")