@TAP_deals is a bot on Twitter that uses machine learning to extrapolate the estimated value of a plane based on historical trends from listings of all other planes on Trade-a-Plane.com. While the code underpinning the Twitter bot constantly evaluates all planes listed at Trade-a-Plane.com, it only Tweets out planes that are undervalued at 5% or more of the list price in order to surface only the most interesting listings. For the data folks out there, The R² of the predictions that the TAP_deals bot generates hovers around 0.77, and the model is particularly predictive for low cost single and multi-engine piston planes with two to six seats (since there’s many observations in this part of the market). To get a better sense of the current state of the @TAP_deals bot, read on!
In many cases, the listing price of a plane is aspirational on the part of the seller — there’s of course many serious sellers with realistic prices, or companies selling planes as a business, but there’s just as many folks “testing the waters” for selling their plane. I recall a story I read online once where someone had a plane listed for ≈$40k but ended up selling for $20k cash no questions asked on a whim — in my own experience, I’ve talked to sellers arguing for $37,000 who flipped to $30,000 as soon as they thought they may have me on the hook, but I was losing interest. Because of this, the goal of this model is to, as accurately as possible, reflect the actual value of a plane instead of working off of the listing price — while a model that perfectly reflected listing prices would be potentially interesting, it’s certainly not nearly as useful as one that predicts the value of a plane relative to how the rest of the market operates.
Pricing Model
Log(Price) predictions for single engine piston planes from Piper, Cessna, Beechcraft, and Mooney
Price predictions for single engine piston planes from Piper, Cessna, Beechcraft, and Mooney
Currently, there’s about 4,000 planes in the dataset that the model trains on, and each observation is converted into several hundred variables. Some of them are obvious such as the make, model, year, SMOH, and so forth — these are the core characteristics of planes and account for a huge amount of the variation of the net value. In cases where listings may not specify things such as SMOH, total hours, the year of the plane and so forth, those numbers are imputed by looking at other similar planes and choosing the average value for planes matching the other known characteristics of the plane.
Avionics Matching
Beyond the “bare bones” of a plane, another huge driver for cost is what’s inside the cockpit. In the part of the market I pay most attention to (Cherokee 180’s and Beechcraft Sundowners/Sierra’s), the difference between a bare-bones stock plane from the 1960’s and a fully updated cockpit can be the difference between a $28,000 plane and a $78,000 plane. Obviously, this is something that is missed when you just take into account the basics of a plane’s stats, and as a result, estimates on the value of a plane are going to be way off if these aren’t taken into account. There’s one huge problem with accounting for this information, however — on Trade-a-Plane, it’s represented as a free-form text box — here’s the equipment list from a plane, for example:
100% free form, 100% free-for-all
Obviously, this is structured data that fits a certain pattern — if we had a list of all of the avionics in the world, we could simply just match these up with our canonical records and be done with the problem. Since it’s a free form text box, though, we also have sellers that represent avionics in different list styles, in shorter model-number-only representations, and so forth
In short, we need to be robust to these variations. Internally, the avionics matching code consists of several phases — first, we maintain a list of ten thousand avionics that we can match listings against. Next, we generate dozens of permutations of ways someone could write the name of an avionic, then, we generate similarity features between a given listed avionic from a post and each of these permutations, and then, we pass these features through a model that has been internally scored and trained to specifically identify the correct avionic. The internal dashboard used for generating these picks looks like this page:
Internal vetting for matching free-form avionic listings to canonical avionic references
Once matched, we now have a “canonical” avionic reference for the free form text — we can now directly compare two planes as a result (i.e. we know that both of these planes have GPSes, one a Garmin brand and one not — what is the marginal increase in listed price for the Garmin brand name above Apollo?). In practice, this avionics matching code can correctly match free-form text to the actual avionic in about 84% of the cases we observe.
Once we have identified the core characteristics of a listing, imputed the missing characteristics, and matched all the avionics we can possibly match, we’ve now fully cleaned the listing. We can now take this cleaned listing and place it in a larger model that predicts the price of the plane. The target label that we try to predict is, of course, the listed price of the plane. The resulting estimate, then reflects the best guess at the value of a plane given the general trends for planes in the Trade-a-Plane database. Put together, the full flow of our software looks roughly as follows:
Flowchart for how we estimate the price of planes on Trade-a-Plane
To the extent that our price differs from the listed price, this represents the degree to which the plane appears to be over or under-valued. Of course, this is our opinion about the price rather than ground truth about the degree to which the plane is off-base on it’s listing price. A plane is worth however much money a buyer will pay for it. The estimate our model provides, however, comes from a principled data-science perspective, is generally in line with most prices of most planes, and when it differs, from qualitative review, a fair number of those differences look to be picking up on some interesting trends. Next steps for this system include improving the accuracy of the avionics matcher, re-targeting labels to only planes that appear to have been sold rather than all planes listed (i.e. reflect likely sale price rather than listing price), and toying with increasingly sophisticated ML models for estimating the plane price. In the meantime, however, we will continue to list out planes that seem undervalued in order to cut down on spending way too much time searching through listings.