I don't know if there is any "value" to it but I think it is an interesting concept!

I've actually thought a lot about this very thing as part of my research. Background, I did some stuff in the miltiary using cool software to analyze things and I took that approach to my research. Not that I have the software (the civilian license is several thousand per pop), but I used some of the aha moments I developed fighting bad guys in some foriegn country in my approach to writing about filibusters in Texas. (After all, how different in terms of general patterns, supply, logistics, etc. is an insurgent army supplied by an external influencer in Texas in 1813 from insurgent group X in country Y supplied and supported from country Z in 2008? The answer is, surprisingly not as much as you'd think.)

1. Maps are key. I took a bunch of datapoints on letters in my book research project and mapped things out just to understand it. The first thing I did was identify that some letters attributed to one person could actually not have been him, and from that, discovered some nuances in handwriting that led me to identify a bunch of his brother's letters which were mixed in with his, and therefore change the narrative (these were their own personal copies of their letters and hence, not signed).

I mapped out one guy's travels through Mexico on Google Earth, and that helped me visualize where he went and what he was doing. I got to thinking a lot about this and the implications. For example, there are already guys who sit there combing through WWII mission records and figuring out that Japanese Pilot X shot down American pilot Y or that the submarine spotted and sunk on Day X was U-85, or whatever. Or Pirate Blackbeard was in this area in this week, and what is the likely radius of his ship, and if some ship disappeared, could he have taken it. Pick your problem, you can see implications.

2. But here's where it has a bunch of potential to go much further than that. The problem is actually two problems: organizing the information, which is what you're talking about, and identifying, cateloguing and going through that information. Now, we have all of Lincoln's letters, or most of them. But historical info is hidden in a thousand different places, and no one, not even the archives that have these documents, knows what's in them all. This is where technology can blow this up. Lincoln historians have poured through his letters and those of the generals and politicians. But imagine if you could access every letter ever found that mentions Lincoln in connection with July 3, 1863. Say a private wrote a letter and said. "I saw Mr. Lincoln riding by with General McClellan this morning at dawn." Or whatever. This is of course impossible to do in a lifetime...traditionally. Tech is changing this.

3. Google is rapidly digitizing books, and libraries are digitizing records. Once it's in a digital form, you have the ability to use software to pluck out these datasets. But of course, it's a huge mountain of information. Now, imagine you turn to AI to attack this problem. Let's take the WWII air missions as an example. You get all of these documents in digitized format, then you teach the computer to go hunt for information, extract it and put it into databases. You even teach it to detect anomolies and flag it. I've heard of shipwreck hunters who searched for years on one bearing and then said, hey, maybe that number was transposed and changed their search grid and found the ship. Well, the computer can work all that out for you. It can tell the average speed of a ****e-Wulf 190, and possibly even a specific one or a pilot based on his typical flight pattern, and say, "the likelihood of him being at point X is 85 percent, while the likelihood that the other guy being here is 35 percent"

4. Now, let's add in something new, technologically which I think will be a groundbreaker: digitization of cursive.

You see, so many of these old records that I spent hours tracking down in the national archives are handwritten. Previous historians would sit there and burn up precious archive time trying to decipher them. They're often damaged or hard to read, etc. What I did was just photograph the hell out of all the documents, using different light settings on the microfilm, or getting close up documents that I had the originals of and looking at them very closely. (I held Zebulon Pike's orderly book like this, and by the way, it still smells of campfire smoke). Here's an idea of what I was dealing with: On the left is a letter in the National Archives written by American special diplomatic agent William Shaler, who plays a role in the revolution in Texas in 1813. This is the letter he sent to James Monroe. On the right is his own copy of the same letter, preserved in his letterbooks.


(Edit: My images aren't showing up but it should be one very dark, almost illegible letter and one very clean letter on the right)

You can see there's a really big difference. The first is microfilmed, and depending on the original, it may or may not be legible at all. Sometimes you can sit there in the microfilm room at the national archives and play with the lighting settings. But that's very time consuming. Now imagine you can get a computer to do that for you, and read the words. And of course, in the dark copy, the words from the back side have bled to the front. It's really hard to read by the naked eye. But a computer can tell by slants and other things what is what and basically quiet the background noise and pull up only the front-facing letters, enhance them and make them readable.

5. Here's where AI comes in. The very simplest form of AI is spell check. It digitizes the letter, and if words are undecipherable, can use its algorithms to posit presumptive words that fit the letters you can see, then check them for grammar or usage to see if they fit. Then it provides not just an answer, but a range of answers. A good example from my research. I read an article by a historian in the 1940s that has since been cited by other sources, which themselves are cited (you can see how echo chambers begin), in which a secret spy is identified as a "Mr. Quist." But there was no such person that I could find. However, there is a guy in my story named "Quirk." So I thought, maybe this person writing in the 1940s misread the manuscript and guessed. And maybe 80 years of historians have been overlooking something because they didn't double check. So I double checked. The text is indeed blurry and it could be either one. But I verified that Quist is not correct, which along with my circumstantial evidence, makes Quirk an almost certain match.

Now, if you're a computer, you can do that, and when you come across something confusing or ambiguous, you can have multiple answers, as well as hyperlinks to the original researcher. Or, you can flag it, and your AI bot learns to read Mr. Smith's peculiar handwriting better, and the more letters it reads, the better it is able to transcribe it.

6. Going through old files and digitizing them is ideal, but it's also very time consuming. But we already did this in the 1940s and 50s with microfilm. Take the microfilm rolls and spin them through a digital reader. Not at human speed, but machine speed, and film them into digital form (the National archives has already done a lot of this) and then you generate your documents, from which you can pull your data.

7. The last step is an AI machine that can sort through this mound of information, populated in databases, and extract info. Every note is geotagged. Every name is put in a database. Every name is tagged with possible verifiable people. The Census is absorbed into this borg-like database and the AI bot can tell every George Wood who is a tailor born in England, for example. The database figures out his genealogy. So when George writes that he visited Penobscot Maine, it can tell that he has a cousin there, and that cousin is a fisherman, and if he says that he heard a rumor that Abe Lincoln liked to swim naked with McLellan's wife, then we can say hey, that's a valuable connection. (Or whatever, I'm just making up examples).



Basically, you turn this into an all-seeing eye of Sauron on history. Now, that's not going to put historians out of business. My research showed just the opposite. Using some of these sort of techniques (minus the AI) on a small scale level made me realize that the need for interpretation of the data actually increases, not decreases as you expand this.