For the October 7 meeting at 4:00 pm EST.
I'm going to start with my apologies for having the agenda ready so shortly before the meeting, but since we'll be covering some of the points of last meeting's agenda, which got cancelled, you should be able to recognize a couple things.
Today we have a special guest, long time Thwinker, joining the core group meeting: William Kurkian. Another special thing about the meeting, is that we will try to make the best out of the opportunity of having so many sharp and creative minds sitting together, and have this be more of a working meeting, rather than only a reporting meeting. Once again, this meeting is divided in two parts. The first one is organized by one section per person, so that (in Jack's words: ) "each of us can report on our individual responsibilities or areas of interest as well as pose penetrating questions and observations about anything".
The second part of the meeting will cover group topics.
In order to first close the big and challenging chapter on how to solve the problem of setting rule's CLs before we start a new one, we will start with Jack's report:
A. Jack
1. Report on how to implement Bayes Rule. How to do this has been figured out, so the problem is solved. There were several interesting discoveries. The first was that deductive rules can use the present approach to calculating a conclusion's truth level, while inductive rules need to use Bayes Rule. The second discovery was that decision chaining can work with a single piece of data per input, its truth level. This greatly simplifies the tool. The third discovery was that deductive rule inputs have weights, while inductive rule inputs need importance. Weights and importance behave very differently and so need different terms. The next step is to implement the software changes. This is started and is going very well.
2. Once the software changes are done, one possible area I could work on is doing some claim-checks that are NOT based on fact-checks. We have quite a risk here, as we don't know what the "unknown unknowns" are. Montserrat, what do you thwink of this? How does it fit into your project plan?
3. Our conversations with Michael Hoefer, a potential new thwinker, continue to go well. We had about a one hour Skype call on Saturday with Michael, Jack, and Montserrat. Lots of fascinating topics were covered. The main one was a line of research that involves using SIP and system dynamics modeling to analyze the hate-based authoritarian problem. Michael is keen on doing this, as if fits his skills and interests well. Jack will create a new section on the forum for the project.
(A new thread has been opened, where the most relevant email exchanges with Michael can be found)
B. Montserrat
1. The next problem asking for a high quality solution, is the question of how to set the fact's CLs. I have been working on this during the last week. My plan has several steps. I started by learning how other fields approach the same problem. Good ideas are likely to be found in the field of law, journalism (which obviously includes fact-checking organizations), and maybe even science, history, and Wikipedia.
Interesting findings until now have been that in legal procedures there are indeed different categories of evidence, and the decision of what evidence must and must not be considered is based on what is known as the "rules of evidence", I will be investigating that further in the days to come.
All further steps are on the document I'm preparing, which I will publish on the forum, as soon as I get Jacks feedback on it (I don't want to confuse the readers more than necessary
)
2. I discovered that we may have to start referring to the "fact" nodes differently. Here's a short quote from the document I've been working on, on "How to set fact CLs":
There are some ideas already on possible new labels for "facts". I thought about the following:
- Fact for facts with 100% CL
- False fact for facts with 0% CL
- Truth proposition for facts with a CL between 0 and 100%
Jack thought about the following:
- Truthity / Certainty for facts with a 100% CL
- Falsity for facts with a 0% CL
- Factum for facts with a CL between 0 and 100%
Originally I wanted to use this section to discuss the different names, and other ideas, but turns out that is something we won't be able to decide until we make sure that those three are the actual categories for facts. The question of what types of facts are there may be at the very crux of the problem. About this, there are also some thoughts in progress, I also think that for the categorization, it may become very relevant making the distinction between the dimensions of a fact's actual truth level and a fact's truth confidence level (or truth probability). The reason why this is the case, is because we may find that the categories of the truth confidence levels don't fit 1:1 the categories of different types of claims. For example, we may determine that facts are boolean, meaning that they can be either completely true or completely false, but that for our purposes of calculating the truth level of a claim, having a category of truth CLs between 100% and 0% is useful anyways.
3. If the distinction between the real truth level and the truth confidence level of a fact turns out to be relevant, we will also need to agree on the way we will be referring to those two different concepts. I used to think that the label "confidence level" could be somewhat confusing if we ever calculate the statistical confidence level (a confidence interval) of our confidence levels. That right there is using the same name for two different things - not a great idea. I thought about referring to CLs just as "truth levels", but if it turns out to be relevant differentiating between the actual truth level of the fact, and the truth confidence level, then the word "confidence" makes sense again. I will make my best to determine if this will be necessary or not.
4. Answer to Jack's question (may already have been covered at this point): I'm not sure we can start doing claim checks until we have solved the problem on how to set fact CLs, since that is the data used for the calculation, but let's discuss this!
5. Jack asked me what he could do to use his time best to help. I'm not quite sure yet, maybe we can discuss this together in the group section.
C. Scott
General questions and observations.
D. William
Review of his work and how he may be able to play a sort of board-of-directors advisory role, like Scott. What areas might he have relevant expertise and interest in?
General questions and observations.
E. Group topics - Discussion of A system for setting probabilities. How can we solve this problem?
Here are some question's from last week's agenda that we can still consider for today's group discussion:
I'm going to start with my apologies for having the agenda ready so shortly before the meeting, but since we'll be covering some of the points of last meeting's agenda, which got cancelled, you should be able to recognize a couple things.
Today we have a special guest, long time Thwinker, joining the core group meeting: William Kurkian. Another special thing about the meeting, is that we will try to make the best out of the opportunity of having so many sharp and creative minds sitting together, and have this be more of a working meeting, rather than only a reporting meeting. Once again, this meeting is divided in two parts. The first one is organized by one section per person, so that (in Jack's words: ) "each of us can report on our individual responsibilities or areas of interest as well as pose penetrating questions and observations about anything".
The second part of the meeting will cover group topics.
In order to first close the big and challenging chapter on how to solve the problem of setting rule's CLs before we start a new one, we will start with Jack's report:
A. Jack
1. Report on how to implement Bayes Rule. How to do this has been figured out, so the problem is solved. There were several interesting discoveries. The first was that deductive rules can use the present approach to calculating a conclusion's truth level, while inductive rules need to use Bayes Rule. The second discovery was that decision chaining can work with a single piece of data per input, its truth level. This greatly simplifies the tool. The third discovery was that deductive rule inputs have weights, while inductive rule inputs need importance. Weights and importance behave very differently and so need different terms. The next step is to implement the software changes. This is started and is going very well.
2. Once the software changes are done, one possible area I could work on is doing some claim-checks that are NOT based on fact-checks. We have quite a risk here, as we don't know what the "unknown unknowns" are. Montserrat, what do you thwink of this? How does it fit into your project plan?
3. Our conversations with Michael Hoefer, a potential new thwinker, continue to go well. We had about a one hour Skype call on Saturday with Michael, Jack, and Montserrat. Lots of fascinating topics were covered. The main one was a line of research that involves using SIP and system dynamics modeling to analyze the hate-based authoritarian problem. Michael is keen on doing this, as if fits his skills and interests well. Jack will create a new section on the forum for the project.
(A new thread has been opened, where the most relevant email exchanges with Michael can be found)
B. Montserrat
1. The next problem asking for a high quality solution, is the question of how to set the fact's CLs. I have been working on this during the last week. My plan has several steps. I started by learning how other fields approach the same problem. Good ideas are likely to be found in the field of law, journalism (which obviously includes fact-checking organizations), and maybe even science, history, and Wikipedia.
Interesting findings until now have been that in legal procedures there are indeed different categories of evidence, and the decision of what evidence must and must not be considered is based on what is known as the "rules of evidence", I will be investigating that further in the days to come.
All further steps are on the document I'm preparing, which I will publish on the forum, as soon as I get Jacks feedback on it (I don't want to confuse the readers more than necessary
)2. I discovered that we may have to start referring to the "fact" nodes differently. Here's a short quote from the document I've been working on, on "How to set fact CLs":
There are some ideas already on possible new labels for "facts". I thought about the following:
- Fact for facts with 100% CL
- False fact for facts with 0% CL
- Truth proposition for facts with a CL between 0 and 100%
Jack thought about the following:
- Truthity / Certainty for facts with a 100% CL
- Falsity for facts with a 0% CL
- Factum for facts with a CL between 0 and 100%
Originally I wanted to use this section to discuss the different names, and other ideas, but turns out that is something we won't be able to decide until we make sure that those three are the actual categories for facts. The question of what types of facts are there may be at the very crux of the problem. About this, there are also some thoughts in progress, I also think that for the categorization, it may become very relevant making the distinction between the dimensions of a fact's actual truth level and a fact's truth confidence level (or truth probability). The reason why this is the case, is because we may find that the categories of the truth confidence levels don't fit 1:1 the categories of different types of claims. For example, we may determine that facts are boolean, meaning that they can be either completely true or completely false, but that for our purposes of calculating the truth level of a claim, having a category of truth CLs between 100% and 0% is useful anyways.
3. If the distinction between the real truth level and the truth confidence level of a fact turns out to be relevant, we will also need to agree on the way we will be referring to those two different concepts. I used to think that the label "confidence level" could be somewhat confusing if we ever calculate the statistical confidence level (a confidence interval) of our confidence levels. That right there is using the same name for two different things - not a great idea. I thought about referring to CLs just as "truth levels", but if it turns out to be relevant differentiating between the actual truth level of the fact, and the truth confidence level, then the word "confidence" makes sense again. I will make my best to determine if this will be necessary or not.
4. Answer to Jack's question (may already have been covered at this point): I'm not sure we can start doing claim checks until we have solved the problem on how to set fact CLs, since that is the data used for the calculation, but let's discuss this!
5. Jack asked me what he could do to use his time best to help. I'm not quite sure yet, maybe we can discuss this together in the group section.
C. Scott
General questions and observations.
D. William
Review of his work and how he may be able to play a sort of board-of-directors advisory role, like Scott. What areas might he have relevant expertise and interest in?
General questions and observations.
E. Group topics - Discussion of A system for setting probabilities. How can we solve this problem?
Here are some question's from last week's agenda that we can still consider for today's group discussion:
Last edited:
