A Brief Introduction to How the PPKB Works

Basic Terms

A text is anything someone has written or said, in a speech, article, video, conversation, etc. The term "text" signifies the discourse has been captured as text and is ready to be analyzed. A text contains one or more arguments.

The standard definition of an argument is one or more premises supporting a conclusion. The PPKB, in order to add analytical rigor, takes this a little further. A PPKB argument consists of a conclusion, at least one factual, and at least one rule of inference. A factual is a fact, premise, intermediate conclusion, or main conclusion. A rule states how a conclusion follows from its factuals. Thus another definition of a factual is any argument component that's not a rule.

A PPKB argument may be simple, with only one conclusion, or it may be complex, with multiple premises, intermediate conclusions, and a final main conclusion. Most real world arguments are complex.

In the PPKB, texts are analyzed by identifying their arguments and analyzing each argument. Each argument makes a claim, which is the argument's main conclusion. This receives an automatically calculated confidence level of 0% to 100%, which measures how much confidence the analyst has that the claim is true. The weighted average of the argument confidence levels become the confidence level for the text.

A weight is the relative importance given to each argument in a text. This is necessary since some arguments matter very little, while others are of central importance. For example, in a television ad the argument "I'm Obama, and I approve this message" implies this is a legitimate ad. It's of low importance so it receives a low weight. Weights are also used in argument elements, as explained below.

Purpose

The purpose of structuring an argument this way is to allow arguments of any kind to be scientifically analyzed in a replicable, sound manner that comes as close as possible to how we think, without getting overly complex. The goal of this approach is to allow every component in an argument to have a confidence level that is scientifically calculated. If this can be done, and so far it looks like it can, then citizens will at long last have a reliable guide to how much trust they can place in the truth of sources they must be able to trust, such as politicians, news sources, authors, and political parties.

The PPKB can analyze arguments of any kind, so it can produce Truth Ratings of any kind, by taking a random sample of arguments (claims) to be analyzed from a particular source. Their average rating is the Truth Rating for that source.

Once Truth Ratings are widely available for sources the public must be able to trust, especially politicians, democratic societies will pass through a mode change. Currently most democracies, particularly the United States, are trapped in the dominant Race to the Bottom among Politicians mode as described in the Dueling Loops paper. But once Politician Truth Ratings become established, average political truth literacy will rise very significantly, from low to high. This will cause loop dominance to shift from the Race to the Bottom to the Race to the Top among Politicians. And it will stay there if additional solution elements that also push on the high leverage point of raise truth literacy are also implemented.

A long term Race to the Top among Politicians in modern democracies has never been seen before. None have lasted longer than fifty years or so, as seen by the near constant stream of political bickering, cyclic changes of which party is in power, conflict, revolution, and war that has plagued mankind every since the first political system appeared long ago.

The Thwink analysis of the change resistance subproblem produced nine sample solution elements for pushing on the high leverage point of raise political truth literacy. The reason we are implementing Politician Truth Ratings first is it has the biggest bang for the buck. It appears to require the smallest amount of effort to achieve the greatest impact. We say this because we have seen the high beneficial sustained impact of ratings in various industries, like credit ratings, restaurant ratings, bond ratings, and product ratings like those done by Consumer Reports.

A Simple Classic Argument

Here's the standard classic argument used to illustrate what an argument is:

1. All men are mortal.
2. Socrates is a man.
3. Therefore Socrates is mortal.

Elements 1 and 2 are premises. Element 3 is the conclusion. The conclusion logically follows from the premises by application of a rule of inference called Modus ponens: "If P, then Q. P. Therefore Q." Here's how this rule applies: Element 1, "All men are mortal," is the same as "If the object is a man then it is mortal." This may be abstracted as "If P, then Q." Element 2, "Socrates is a man," may be abstracted as "P." Element 3, "Therefore Socrates is mortal," follows logically.

Take the time to examine how this rule of inference applies to elements 1, 2, and 3. If you can clearly grasp how this works and start thinking this way when you examine arguments, then you will find the PPKB fairly easy to use. Don't worry if you have to read this section several times, because we normally don't think this way. But if we want to be able to correctly and efficiently learn how to analyze arguments, we must learn how to think this way. There is no alternative.

Arguments quickly become so complex that the human mind becomes unable to understand how they work and how sound they really are. This forces the mind to resort to intuition, which is unreliable. To reduce this tendency the PPKB automatically generates a diagram (or map) of the structure of each argument. The diagram for the above argument looks like this:

Socrates is Mortal Diagram

Boxes 3, 6, and 8 correspond to elements 1, 2, and 3 in the original argument above. Box 7 is the Modus ponens rule of inference. Boxes 1, 2, 4, and 5 have been added to make the analysis complete rather than partial. How do we know "All men are mortal"? How do we know "Socrates is a man"? In rigorous argument analysis we must take these premises and go back to the real world facts they are based on. Boxes 1, 2, 4, and 5 show how this may be done. It's extra work. But it takes your analysis from fluff to concrete, from unreliable to reliable.

Note how much information the diagram contains. Factuals (facts, premises, or conclusions) are green, rules are yellow. Each element has a confidence level (CL). If more than one factual or more than one rule is used to reach a conclusion, they have weights. (For simplicity these are 50% for the two premises.) The diagram is designed to be read rapidly, which you will frequently do as you analyze arguments. Compare the PPKB argument diagram to other argument maps.

Finally, here's how the fully analyzed argument looks:

Argument Diagram - Socrates Is Mortal

We could have designed the PPKB to not use rows. An argument would be analyzed by drawing its diagram. While more user friendly, this is a formidable programming task. To get the first generation of the PPKB out the door we elected to take the rows and separate diagram approach. This gives us quick proof-of-concept that the tool can work.

How to Analyze a Text

Suppose you have a text. Here's how to analyze it using the PPKB:

  1. TEXT GROUP - Add a new Text Group if necessary. A Text Group contains a group of texts, such as the TV ads for a particular campaign or some speeches. Each ad or speech is a text. Text Groups are useful for comparing the analysis results of one group of texts to other groups, as well as analyzing the results of the texts in a group.
  2. TEXT - Add a new Text to a Text Group. This is the text you want to analyze.
  3. ORIGINAL TEXT TAB - In the Text form, click the Original Text tab. Copy and paste the text into the one large textbox. Let's use this actual text from a U.S. TV ad as an example:
  4. Obama: Those ads, taking my words about small business out of context, they’re flat out wrong. Of course Americans build their own businesses. Every day, hardworking people sacrifice to meet a payroll, create jobs, and make our economy run. And what I said was that we need to stand behind them, as America always has, by investing in education and training, roads and bridges, research and technology. I’m Barack Obama, and I approve this message because I believe we’re all in this together.

  5. MARKED TEXT TAB - Click the Marked Text tab. Copy and paste the same text into the one large text box. Then study it closely. Understand what the text is trying to say and how it makes its case. Identify where the argument elements are using marks like "(Premise)" and "(Conclusion)." There is usually more than one argument unless the text is very short. For multiple arguments use marks like (1 Conclusion) and (2 Conclusion). Start by identifying the conclusions first. For example:
  6. Obama: Those ads, taking my words about small business out of context, (1 Conclusion) they’re flat out wrong. Of course Americans build their own businesses. Every day, hardworking people sacrifice to meet a payroll, create jobs, and make our economy run. And what I said was that we need to stand behind them, as America always has, by investing in education and training, roads and bridges, research and technology. (2 Conclusion) I’m Barack Obama, and I approve this message because I believe (3 Conclusion) we’re all in this together.

    This is a tricky text to analyze. An argument consists of one or more premises supporting a conclusion. In this text the arguments overlap and are scattered about. Their components are not all present but are implied. This makes political text analysis challenging. But if you can clearly identify the conclusions, then you can make a reasonable analysis.

    The above box shows three conclusions identified. The next step is to identify the premises:

    Obama: (1 Premise) Those ads, taking my words about small business out of context, (1 Conclusion) they’re flat out wrong. (1 Premise) Of course Americans build their own businesses. (1 Premise) Every day, hardworking people sacrifice to meet a payroll, create jobs, and make our economy run. And what I said was that (3 Premise) we need to stand behind them, as America always has, by investing in education and training, roads and bridges, research and technology. (2 Conclusion, premises implied) I’m Barack Obama, and I approve this message because I believe (3 Conclusion) we’re all in this together.

    The premises and conclusions are identified. It's still hard to understand, so let's reorganize it into something more understandable. Let's also compress the markings to abbreviations, like (1 P) instead of (1 Premise).

    Obama: (1 P) Those ads, taking my words about small business out of context, (1 C) they’re flat out wrong.

    (1 P) Of course Americans build their own businesses.

    (1 P) Every day, hardworking people sacrifice to meet a payroll, create jobs, and make our economy run.

    (1 P) And what I said was that we need to stand behind them, as America always has, by investing in education and training, roads and bridges, research and technology.

    (2 C) I’m Barack Obama, and I approve this message because I believe (3 C) we’re all in this together.

  7. STRUCTURED ARGUMENTS TAB - Copy the marked text. Click on the Structured Arguments tab. Paste. Now comes the hardest part of the analysis. Rearrange the text so that each argument is listed separately. Add and clarify as necessary. Structure each argument by having one paragraph for each element. Each argument should have one main conclusion at the bottom. "IC" means intermediate conclusion. The example uses some abbreviations and some full words, to illustrate that you can take the approach that works best for you.
  8. As you can see, this is a tremendous amount of intellectual work. We are working on ways to make it easier.

    Main argument:
    P - Every day, hardworking people sacrifice to meet a payroll, create jobs, and make our economy run.
    Therefore
    IC - Americans do build their own businesses.

    P - My words were taken out of context.
    P - What I said was that we need to stand behind them, as America always has, by investing in education and training, roads and bridges, research and technology.

    Therefore:
    Those ads saying Americans don't build their own businesses are wrong.

    Minor argument:
    A standard reusable argument structure. The claim is generic.

    Fact - If the visuals match these photos and videos of Barack Obama then this is Barack Obama. 95% (If P then Q)
    Fact - This visual instance matches the fact photos and videos. 95% (P)
    Rule - Modus ponens. 100%
    Premise - This really is Barack Obama. 95% (Q) (P for next persuader)

    Rule - If this really is the identified speaker, then he or she spoke the text. 100% (If P then Q)
    Premise - "I’m Barack Obama, and I approve this message...."(Q) (P for MC)

    Fact - If the speaker approved this message, then this is a legitimate statement. 100% (If P then Q)
    Rule - Modus ponens. 100%
    Main Conclusion - This is a legitimate ad rather than one from an unidentified source with usually devious intentions. ??% (Q)

    Minor argument:
    P - What I said was that we need to stand behind them, as America always has, by investing in education and training, roads and bridges, research and technology.
    P - We live in a socially dependent world. No single person can thrive without the support of a much larger group, which provides the infrastructure needed for the group to thrive.
    Therefore
    We're all in this together, i.e. the people and the government are working together.

  9. ADD ARGUMENT ELEMENTS - Now for the fun part, where the magic happens. Add one argument row to the text for each argument in Structured Arguments. Copy the text for each argument and paste it into the row.
  10. For each argument row, click on the Argument button. This takes you to the Argument form. Add one row for each element in your structured argument. Take your time, because there will be considerable discovery and evolution until you have analyzed many arguments. Add facts and persuaders to the database as necessary, though this should be minimized to avoid clutter. Give each element except the main conclusion its correct weight if necessary.
  11. CALCULATE - When all elements are entered click Calculate. This will validate your argument rows and give helpful feedback if errors are encountered. If an error is found, fix the problem and click Calculate again. Calculate also automatically updates the diagram.
  12. Below are the final results for the three arguments in the text. Notice how they differ from the structured text. As the three arguments were entered and diagrammed, the analyst thought of a better way to analyze some of them. Also note how there's a full paper trail leading from original text to the final analyzed argument.

    The diagram for argument 2 is small and hard to read, so here's the full diagram. This is what you would see if you clicked on Show Diagram.

    Argument 1

    Argument 2

    Argument 3

    The PPKB organizes argument data into a hierarchy. At the bottom (level 4) are the facts and rules used to compose arguments. Above that (level 3) are the arguments. Above that (level 2) are texts. If you click on the Text button of one of the above arguments, it takes you to the text tab as shown below:

    Argument text

    The text tab shows that the three arguments, when weighted to show their relative importance, have a weighted confidence level of 96%. The rules used have a 100% confidence level, indicating no fallacies were used. That's a good sign this politician leans towards the truth. The factuals have a 96% confidence level. That's pretty good.

    Clicking on the Group button takes you to the Group tab (level 1) as shown below:

    Group

    The first row is the text we just analyzed, with a confidence level (CL) of 96%. The other two texts have lower CLs of 28% and 69%. The three texts have an average CL of 64%.

    To select a different group, click on the Select Group button. Here's what you would see:

    Select Group

    We've been working on row two, TV ads for the US presidential campaigns from 1952 to 2012, democratic.

    The above images show how the PPKB organizes argument data into a four level hierarchy:

    Level 1 - Groups
    Level 2 - Texts
    Level 3 - Arguments
    Level 4 - Facts and rules

  13. REVIEW - When all errors are fixed, calculation results for the argument you are working on will appear. Check them closely. Do they make sense? Do they reveal any bias? Could you confidently stand behind these results and justify every decision you made in each argument row? Have you reused as many facts, rules, and other arguments as possible, rather than adding new ones?
  14. Most importantly, do you think that other analysts would come to about the same argument confidence level as you have and that the confidence level comes close to reality?

  15. Done - If the answer to all these questions is yes, congratulations. You have made a vital contribution to the Political Persuasion Knowledge Base.

Politician Truth Ratings

To create a Politician Truth Rating we would create a new group for that politician. Then we would randomly select important claims that politician had made over a period of time. Each claim would be a text. Most texts would probably have just one argument. But some, since the claim is so complex, would have multiple arguments. Once all claims were analyzed, the confidence level for the group would be the Politician Truth Rating.

In a similar manner the PPKB can be used to create Truth Ratings for other sources, such as authors, publications, organizations, political parties, and even events, like a summit, meeting, or convention. The PPKB is not limited to just politicians, because they are not the only sources of information that the public must be able to trust to tell the truth.