June 2018 - Core Group Meetings

Jack Harich

Administrator
Staff member
#1
Here's the agenda for our June 3rd meeting at 8:00PM EST.

A. Montserrat
  1. Update on accommodations for my stay in Atlanta.
  2. Anything else?

B. Scott
  1. To be provided.

C. Jack
  1. Report on prototype progress. It's going extremely well. This week I worked on lots of small details and bugs, added the Help doc, activated Log On (which is optional), and improved the way the pages look. It's looking fairly complete. The astounding thing is it's so complete it can do fact-checks, including adding, editing, and using new rules, facts, and reusuable claims. I had not foreseen getting to this point so soon. This is very good news.
  2. Have selected my first fact-check article to turn into a claim-check. Here is the article. I looked at several on Politifact's home page and noticed that the logic in all of them is loose and usually full of small errors. To me this means I've been thwinking in terms of Structured Argument Analysis for so long I can instantly spot argument logic errors. The article I selected has a typical error. The author, in his very first sentence, states what the implied claim is. Then he shows why it's false. At the bottom in the Our Ruling section, he forgets about the implied claim entirely and concludes the claim is mostly true. It's not. I expect a claim-check will show it's mostly false. This is amazing! But I suspect it will be typical because others are working without the tool they need.
  3. Got the new idea that after doing a claim-check of a fact-check, we can contact the author. This is a wonderfully natural way of spreading the word about what we're doing. Of course I'm very curious about how the first such contact will react. We will have to be gentle, diplomatic, and helpful.
  4. Related to the above, as soon as the tool is running smoothly I and perhaps others should take the fact-check training. This will also serve to introduce us to the fact-check community.
  5. Although the prototype is ready for us to use, I'm still finding bugs and usability problems. Let's go another week before starting the test claim-checks experiment. This is briefly described on the prototype Help page in the last item, Prototype limitations and testing.
  6. I've concluded that we need to build out the prototype to have a real database as soon as possible. The prototype looks so complete people will expect and try to use it for real work. Their work will frequently be lost. That could even happen to us, though I hope it doesn't. Since it can happen, we will have to backup our work frequently. Plus doing simulated database updates is fraught with bug potential. I've located a new company in Atlanta that seems more tuned to our needs here than Applied Imagination and offers "Hire an individual expert." I will be talking to them a little more this coming week. Right now I'm estimated the build out will run about 10K and that I have two solid weeks of work to do before they can begin. This is very serious work and I'm trying hard to do a good job.
  7. I was unable to solve the problem of the prototype shortcut keys don't work on Chrome on the Mac. Montserrat and I ran a test, but it didn't give me enough info to solve the problem. I need an actual Mac so I can go into the code and see where the problem is occurring. To do this I will be purchasing a very old used Mac.
  8. This morning I got the idea that in our test claim-checks experiment, we can also collect data on how long it takes each of us to create a claim-check. One of our top two risks is it takes too long. This data will begin to help us resolve that risk. This data is also something Montserrat can possibly use in her paper.
  9. One more item! It occurred to me that Montse's problem with shortcut keys not working on the Mac with Chrome may not be that big a problem for now. As I recall it was only the Alt key that wasn't working. Looking at the Analysis page dropdown, none of the Alt actions that are dimmed are crucial. Thus I thwink we can wait until you arrive to fix this problem. That will save the purchase of a used Mac!
 
Last edited:

Jack Harich

Administrator
Staff member
#2
Scott and Montserrat, any suggestions for tomorrow's agenda? This will be our monthly meeting at the Tower at 2:00PM EST.
 

Jack Harich

Administrator
Staff member
#3
Okay, here goes. Our agenda for June 10, 2018 at 2:00 EST is:

A. Montserrat
  1. It's about time for this, an invite to become a real thwinker and be listed on our contacts page. Would you like a thwink.org email address like Jack and Scott? We need a photo and a bio writeup, which you can update anytime.
  2. It's beginning to look like the claim-check experiment will be your first project and paper. It's a much needed research project that will be pivotal to the success of the Truth Ratings System. Can you start to develop a plan for this project?
  3. Update on accommodations for my stay in Atlanta.
  4. Anything else?

B. Scott
  1. Update on the artificial intelligence course he's taking.
  2. Update on the weighted rankings decision tool he's building.

C. Jack
  1. Using the prototype, on Monday I finished up the sample claim-check article based on a real fact-check article. As I went I fixed bugs and did usability improvements. The number of these has reached such a low level the tool looks ready for general use, which is fantastic news!
  2. For the next three days I studied how to proceed with the next development step, using a real database. This is quite hard to figure out as the technology is so young and in such a high state of flux. It can be easily done by someone who's done it before, but I haven't. Every book, article, and example I study does it differently. So after three days I had gotten almost nowhere on the key aspects of programming. So I put it aside for awhile.
  3. I also continued to look for someone to develop the next step. Lots to say here, but people with full stack Javascript skills who understand single page applications are in short supply. I have found one small company in Atlanta that I plan to call on Monday.
  4. Friday morning I put the finishing touches on the article and the website. Then on Friday afternoon and Saturday I begin contacting people about the Truth Ratings System. So far this has been:
    • Colin Deepen and Politifact, also to get permission to reuse their material.
    • Duke University Reporter's Lab, which does fact-check research.
    • Daniel Funke at the International Fact-Check Network.
    • Long-time thwinker Philip Bangerter, who knows a fact-check org in Australia that he can talk to.
    • The WikiMedia Foundation, who pointed me to the contact below:
    • WikiTribune, a new startup by Jimmy Wales, founder of Wikipedia.
  5. There's some good news here. Stephen Philbrick at the WikiMedia Foundation expressed genuine interest, but there's not a suitable place to post a semi-article about our launch. So he referred me to WikiTribune. I've posted an article there in the form of a suggested project. This looks like a fertile place for such a suggestion, as they really want to get the news industry back on track to telling the truth and are just getting started, so they are looking for innovation.
  6. All in all, we are off to a good start in contacting people about the prototype. Now we wait, see the reaction, and hopefully begin some conversations.
  7. Last night I applied the missing abstraction principle and gap analysis to figure out what it was I needed to learn to move forward on developing the next step. It worked! In one page of writing I moved from feeling lost to feeling like I finally understood how to design a client/server framework for our application. This morning I begin implementing the plan. Again, it worked! Now I have a simple server running that's well encapsulated in a Javascript class. It's the starting point for the framework.
 
Last edited:

Jack Harich

Administrator
Staff member
#4
Hope all's well with everyone! Our agenda for June 17, 2018 at 8:00 PM EST is:

A. Montserrat
  1. It's about time for this, an invite to become a real thwinker and be listed on our contacts page. Would you like a thwink.org email address like Jack and Scott? We need a photo and a bio writeup, which you can update anytime.
  2. It's beginning to look like claim-check experiments is a much needed research project that will be pivotal to the success of the Truth Ratings System. Can we work together and develop a plan for this project so that we can hit the ground running when you arrive? Here are the key strategic aspects I see so far:
    • Study existing papers on fact-checking. Who are the major authors? What are the major issues facing the field? Who looks promising as a co-author?
    • Step A. Test this hypothesis: The truth confidence level of a claim can be accurately measured by use of the right tool, such as Structured Argument Analysis. Here we would repeat claim-checks of the same claim in a feedback loop, until we hit an asymptote. The protocols (and possibly the tool) would be continually improved, allowing better and better accuracy. Once we have good accuracy we can publish a paper and are ready for step B, which is a second paper.
    • Step B. Test these hypotheses: H1. Exposure to claim-check articles significantly raises political truth literacy. H2. Exposure to fact-check articles significantly raises political truth literacy. H3. Exposure to claim-check articles raises political truth literacy more than exposure to fact-check articles by a significant amount. I expect you will rework these hypothesis and may add more.
    • A and B can be done with online surveys. I've run some of these and confirmed they generate useful info. Online surveys have been used in fact-check papers, so we are following the standard approach to experimentation.
    • To perform A we need some good claim-check articles as examples and a reasonable start on the protocols. I thwink we have a starting team of four people to do this: You, Jack, Scott, and Martha. Go team!
  3. Update on accommodations for my stay in Atlanta.
  4. Anything else?

B. Scott
  1. Insights on where artificial intelligence is going and how that might affect Candle and the Truth Ratings System in the long run. (Scott, I'm making a guess here you may have something on this.
  2. Anything else?

C. Jack
  1. Developed a client/server framework to put the prototype into. The framework has a real sql database on the server, handles client request to read from and change records in the database, handles user logon/logoff, and is security hardened by use of an encrypted token the client must pass to the server for all changes to the database, plus stored encrypted passwords. This was a lot of tricky and frequently frustrating work to build. It's been done before many times, but research found nothing I could reuse so I had to build it myself. I hope to make the framework available to others on GitHub.
  2. On Friday I switched to deployment. Who is going to host our application? How am I and additional developers going to deploy the code to the server in a reliable manner? After examining Rackspace, Heroku, and Google Cloud and being quite disappointed, I have gone with Microsoft Azure. Azure is integrated with Visual Code Studio (an integrated development environment, an IDE) so well that deployment is easy. Actual server update time is a fantastic one minute! That means the server is down for one minute while updating, if I understand this correctly. Actually it may be down only for seconds, depending on how they do the rollover from the old to the new running server instance.
  3. But the icing on the cake is the ability to debug running server code in the IDE. I'm close to getting this feature working, which will be a large productivity boost.
  4. Once I get some problems worked out, the framework will be running at this test url. With luck I will have it running before our meeting.
  5. Ben Davis, who was formerly working on the Kickstarter project for raising funds for the Truth Ratings System, reports he's got a great job in north Atlanta. He's working with a startup in an incubator. How exciting!
  6. >>>---> Now for the most important item on our agenda. No response from the fact-check community. Sigh. I've seen this before. Numerous times I thought that what I and Thwink collaborators had created would work to get others interested in the Thwink approach. But it failed, due to what I suspect is new paradigm change resistance and to a lesser extent, lack of proof the approach works and a poor approach to selling the concept. Failure examples are:
    • An analysis and mailout to all 140 Sierra Club chapters just before their national conference, 2005.
    • An article in the Club of Rome newsletter, which goes out to people in over 40 countries, 2006.
    • The change resistance paper, published in an A journal in 2010.
    • A conference paper and huge acceptance of root cause analysis by a workshop leader at a sustainability conference in 2012.
    • The Democracy in Crisis film series, 2016.
    • The Thwink.org website itself. This has attracted some tremendous individuals, but it hasn't led to anything else.
  7. So what should we do? Let's discuss this. Be thwinking of strategic insights on a good path forward....
 
Last edited:

Jack Harich

Administrator
Staff member
#6
Another week has rolled by in a flash! Our agenda for June 24, 2018 at 8:00 PM EST is:

A. Montserrat
  1. To be determined.

B. Scott
  1. Thought you might find this article on IBM's Project Debate interesting. IBM might beat us to Candle.
  2. Anything else?

C. Jack
  1. I focused on trying to deploy the application framework. This proved to be surprisingly difficult. I've still not succeeded. The hosts are very complex, very under-documented, and buggy. I thought I'd found one, Zeit Now, that would work, but they are lacking a key feature: the ability to connect to a database. On Saturday I will be starting to try Microsoft Azure for a second time to see if I can get it to work. Overall, this has been an exhausting task. It's like reading graduate level textbooks without having taken the prerequisites. I'm not a professional sys-op with over ten years experience, which is who normally deploys applications. So there is a gigantically steep learning curve that has been grinding me down. I hope to prevail.
  2. I've been studying a book on Node and Express, the two frameworks our framework uses. The result has been small amounts of improvement to our app and a greater understanding of apps like ours in general.
  3. Tuesday evening I attended a node Meetup group. It was very worthwhile. Afterward we socialized at a restaurant. Stimulating conversations! I will start going to these meetings every month.
  4. I'm just starting to try to spread the word and find a room for Montserrat. Will report on progress.
  5. Had a new idea on closing the Analytical Activists of Atlanta Meetup group and starting one for our central project, Politician Truth Ratings. This may garner us some people to help in experiments and all sorts of things.
  6. But thwinking this over, our experience has been that Meetups don't attract research types. They attract people looking for social interaction and entertainment, with a slight interest in research. They don't attract any professors or university students at all. Thus we should consider looking elsewhere, like journalism and political science university departments. That looks productive and it's how "real" researchers work.
 
Last edited:

Jack Harich

Administrator
Staff member
#7
Terrific meeting. Montserrat made what I thought was a brilliant suggestion: When we run our claim-check experiment, we can choose fact-check articles from WikiTribune to claim-check and then publish the claim-check on WikiTribune.

This has several advantages:
  1. The articles are open, so we automatically have permission to use them.
  2. We have a place to publish our articles.
  3. Right there, in one place, readers can see the first version (the fact-check) and the second version (the claim-check). Ideally they will notice that the second version does a better job of analyzing the claim. It's more understandable and does a better job of stepping through the argument logic. It's also more educational. It's a better form of journalism.
  4. If our claim-checks prove to be interesting reads and are better reads than the fact-check articles, then Jimmy Wales, the founder of WikiTribune and Wikipedia, will probably become interested in what we're doing.
This gives our claim-check experiment a lot of focus.

Also, Montserrat noticed in their FAQ that they don't have a magic fix for how to talk about controversial topics. If we can demonstrate that we can objectively analyze claims, then we have the fix they are looking for. (These are rough ideas from my notes.)

Thanks, Montserrat!
 
#8
Aaaall right, these were the agenda items I covered in our June 24 meeting, sorry for not having shared them earlier!
  1. Atlanta accommodations: I have found a couple options, but all of them are still a little over my budget, so I haven't closed any deal yet. Thanks Jack for helping me by asking friends of your's in Atlanta!
  2. Questions on the forum: last week I posted some questions both in the Comments on the Politician Truth Ratings Project, and the new Claim-Check Experimentation threads. Thanks for your answers Jack!
  3. Follow-up on some topics of the last weeks:
    • I re-read the article you claim-checked. I think we should post your result (or other claim-checked articles like that) on WikiTRIBUNE.
      Also: did the author of the article respond after you contacted him? I found him on Twitter, he has a verified account. I guess if we contact him via Twitter, we could have some chances of him answering.
      When he tweeted about his own article he commented:
      "@KeithRothfus claims one small bank or credit union disappears in the US each day on average. He’s right — but wrong to blame #DoddFrank"
      Update: If I remember correctly, Jack said during the meeting that he had indeed answered but hadn't shown much interest.
    • I found in WikiTRIBUNE's FAQ that they talk about "advisors". They say:
      the advisors listed currently are friends of Jimmy Wales who have been advising him personally, based on their experience, in some cases for years. They are not making editorial judgments. We’re interested in taking advice from a wide range of people and would love to hear your suggestions for additions to the list.
      I couldn't find who the advisors are on the website, but I guess Jack could aspire to be an advisor of WikiTRIBUNE. Why? - There is a weak spot that they assume:
      Controversial topics are hard. We don’t have a magic fix, but there are some useful social norms and values that can really help a community deal with them. “Assume good faith,” “No personal attacks,” “Don’t push an agenda.” Plus a willingness to ban people who misbehave.
      Also, they recognize that being unbiased is hard, especially in the long term, in other words, they don't want to be unbiased, but they don't have any concrete solution for how they're going to achieve that. The truth ratings system is so transparent, it could be the solution. If after the experiments we can prove to be able to analyze claims scientifically, i.e. objectively, then that would be the magic answer to their problem.
    • We could also become community members of WikiTRIBUNE and start sharing there our experience with PTRs. Given that their content is released under a Creative Commons Attribution, we could start by claim checking their articles and sharing that in their own platform. That way it'll come gradually and applied to topics that they're already working with, and we don't deal with copyrights. All in all, I think WikiTribune is a great platform to spread PTR. Also because it is still a beta project, there is still openness for new ideas and change resistance should be low.
  4. Claim-Check Experiment: I started reading Nyhan & Reifler (2014). It's a good place to start and there is more good material I want to study in the following days. Once I've done that, I'll be ready to work on the research design.
 

Jack Harich

Administrator
Staff member
#9
Thanks for the write up. Some good thwinking here!

1. Accommodations - I had a new idea this morning. Since you haven't found a good room yet and it's hard to search for a good room from Mexico, you could stay at the Tower for awhile until you found a room. It has a bed, a desk, internet, plenty of space, a tiny refrigerator, and a half bathroom. That means a sink and toilet but no shower. There are plenty of eating places in walking distance as well as gyms and such for a shower. Just an idea. I've stayed there many times and it's quite comfortable and inspirational ! (I would offer our house in our spare bedroom, but that would be hard on Martha. She tries too hard to be the perfect hostess.)

2. I think we should post your result (or other claim-checked articles like that) on WikiTRIBUNE (WT). - Yes. This can be part of our plan for regular engagement with WT, like articles, comments, attending their meetings, being an advisor, etc. If we do this regularly and demonstrate good quality, I thwink it will lead to interest from them in what we are doing.

But I'm afraid that doing a good job of this will distract me from focusing on software development, which is very hard. I didn't mention it, but how are we going to make a claim-check article look good using a normal editor? Plus we have not received permission from Politifact to reuse their material, so we have only the False Dilemma article ready to publish. Thus what I'd like to do is postpone publishing until you arrive, which is about when I hope to have the multi-user online database version of the software done.

3. Did the author of the article respond after you contacted him? - Yes. He said "Jack - Have you heard back from the folks at PolitiFact?
I'm a contributor and in no position to grant permission, etc."

4. On proposing Jack as a WT advisor - A super idea. Again, I'd like to postpone this until you arrive.

5. We could also become community members of WikiTRIBUNE and start sharing there our experience with PTRs. Given that their content is released under a Creative Commons Attribution, we could start by claim checking their articles and sharing that in their own platform. That way it'll come gradually and applied to topics that they're already working with, and we don't deal with copyrights. All in all, I think WikiTribune is a great platform to spread PTR. Also because it is still a beta project, there is still openness for new ideas and change resistance should be low. - Yes. This is an excellent perspective.

To these ideas I'd like to mention again contacting journalism and political science university departments.

Thanks!
 
#10
Hi Jack,

1. Thank you very much for the offering! Living at the Tower sounds definitely exciting, and inspirational as you said, but I guess I would leave it as an emergency option. As I mentioned, I do have some options, I just haven't closed any deal yet, but I'm optimistic. Earlier today a girl who was offering a nice option in the Emory/Decatour area just contacted me back, and that will probably be it. I'm just waiting for her to send pictures, and make a decision ;)

2. Yeah, we don't need to rush it, I'd actually just start by opening an account and getting to know more deeply the type of content that is being posted there. We can start posting our results in their platform when we have claim-checked articles from them.

3. Yup, I remembered he hadn't shown much interest...

4. I agree. None of my ideas are for immediate implementation.

5. Sure!
 

Jack Harich

Administrator
Staff member
#11
Earlier today a girl who was offering a nice option in the Emory/Decatour area just contacted me back, and that will probably be it. I'm just waiting for her to send pictures, and make a decision
That's great news. Hope you get this room!