As I mentioned last week, the Orioles have hired Loyola economics professor Stephen Walters to the organization. Walters worked with Dan Duquette previously in Boston, and has been with the Chicago Cubs most recently. CamdenDepot has links to several of his posts at Wages of Wins here, which make for some good reads - I especially enjoyed the comment sections.
After answering some questions I had regarding a paper he had co-authored regarding free agency returns for big and small market clubs, Professor Walters was kind enough to let me ask him a few questions about his baseball work:
1) Since being hired, it seems like Dan Duquette has tried to make some changes to the way the club is run, as well as the product on the field. Do you know how much carry-over there has been in the statistics department from the previous regime?
When Dan took the job, the F.O. already had several brilliant young staffers in place who are very well versed in statistical analysis but also have experience in other areas. Some have been promoted; since I'm still meeting people and learning about the organization, I should let you talk to them directly (or to Dan) about their responsibilities.
2) How much input do primary decision makers on teams actually accept (and use) from their stat people? Has there been a shift more recently in this area? Is it ever difficult to convince people with evidence that might contradict previously held beliefs?
This varies with the decision-maker; I've seen GMs who place zero trust in statistical analysis, and others who give it significant weight, along with traditional information from scouts and other advisers. So, how difficult it is to change minds with stats depends on the person. But questioning stat analysis is not always a bad thing: numbers are just a piece of the puzzle, and sometimes the confidence level they should be accorded is low, sometimes it's high. A good decision-maker has to be aware of the limitations of data as well as its strengths.
3) Relatedly, how much stock have you found people in front-offices put into things like "closer's mentality"? If intangibles have real value, then presumably teams would be willing to pay some extra amount for them over and above wins added? Does that happen to any significant degree in practice?
If you've played any sport at a high level, you know that pressure and the mental approach to the game matter a great deal. That said, most of the guys who have risen very far in the professional ranks are at least tolerably good at coping with pressure. Some differences from player to player in this "intangible" exist, however, and it's not uncommon for such skills to be cited as reasons to draft or acquire someone.
4) The AL East is not the easiest division in baseball to compete in - it's conceivable that a 85-90 win team could still end up in 4th place next year. What kind of effect does that have on the Orioles' decision making process?
Every team should be wary of assigning values to players based on what somebody else is willing to pay. Each team inhabits its own market, and a player's worth in that market should be what guides its decisions. To use a real estate analogy, you shouldn't determine your offer on a house in Glen Burnie by looking at prices in Manhattan. But this is true of everybody, in every division: you have to be careful about how you allocate your payroll and efficient in your search for talent, and try to get the most bang for the bucks you have. I'm not sure our approach in that regard would change if we were in the AL or NL Central.
5) Your previous research indicated that large market teams tend to get a better return on their investments in the free agent market, whereas small market teams tend to come out behind; given that, does it ever make sense for a smaller market team to go after a (higher priced) free agent?
It's definitely sensible to go after FAs in some circumstances – as long as your bid is consistent with your market and competitive position. When smaller-market teams generate red ink on such investments, it's usually because they've gotten carried away in the bidding process and paid a big-market price for a particular talent. It's not uncommon to rationalize that kind of behavior by saying that a player will deliver huge new revenues because he'll be the "face of the franchise" or otherwise enhance marketing opportunities, and have value beyond his production on the field. The classic example was A-Rod's first FA contract with Texas; he wasn't just going to help the team win, but to increase demand for real estate near the ballpark. Within a few years, the Rangers realized they'd been victims of their own marketing hype, and were paying the Yankees to take that contract off their hands.
6) Also, all teams (large or small market) appear to pay about the same rate for a win on the free agent market; why is that, and is there any way for a smaller market team to use that to their advantage?
Well, there's actually considerable variance in "the" market price of an expected win from player to player and even position to position. Some of that reflects differences in risk among players and positions or premiums for particular elite players, while some reflects market timing. Again, all teams (whether in big markets or small) should try to shop efficiently, because money wasted when you make a mistake is going to cost you wins in some other area.
7) When analyzing free agent contracts, it's relatively typical to see people use a set win projection for a player in year 1 and then build in a decline for the length of the contract on the production side, and use a set value of a win in year 1 and then increase that rate by some measure of baseball salary inflation on the money side (X is a 3 win player in 2012 and a win is $5 M in 2012, in 2013 he's a 2.5 win player and a win is $5.3 M, so he should be paid $28.25 M total). Is it important to look at these things in present value terms? That is, discounting the money piece using (presumably) general inflation rates but also discounting (or not) future wins in some way?
In principle, yes. In practice, consideration of inflation and discounting to present values is quantitatively less significant than discounting for injury risk or variance in performance.
8) How much are players/contracts looked at as assets? Is an MVP caliber player making more money than he's "worth" really less valuable than a league average player making the major league minimum?
Conceivably, yes; a "bad" contract definitely reduces a player's value in trade and a "good" one increases it. Again, the issue here is opportunity cost. Money you waste on overpaid players is not available to sign international FAs or hire more scouts, etc., that can help you win.
9) (Bad) Teams will seemingly sometimes sign free agents in an effort to "excite their fanbase"; is there any reason to believe that a "big name" brings (significantly) more to a team than the wins he contributes on the field? Relatedly, is there a certain level at the lower part of the win curve in which investing in the free agent market makes sense to put a non-embarrassing team on the field?
I think there are very few players that people pay to see without regard to how their team is doing. That said, putting a team together involves considering future years as well as the coming season, so sometimes it's appropriate to make long-term investments even when you might not expect to contend immediately.
10) Do teams tend to have hard/pre-set budgets? That is, if a team's stated budget is $100 M and they're already at that level, would they forgo signing a player whose marginal revenue product is greater than his costs?
From what I've heard, it is common for owners to specify payroll targets. As a season unfolds, however, wise owners will revise those targets to accommodate acquisitions that will, as you say, generate revenue in excess of costs.
11) How fungible is money in baseball? If a team passes on signing a free agent, would they then re-invest the "saved" money into the draft or player development or whatnot? If a team passes on signing a free agent in year 1, would they re-invest the "saved" money into signing a free agent in year 3?
Again, wise owners will be adaptable: they will allow savings in one area to be banked for the future or re-allocated in the present. But I haven't worked for enough teams or talked to enough GMs to know how common this is.
12) How much "Picasso" value does having a good baseball team have? Is it unfair for many fans to expect owners to take losses or make economically poor moves in an effort to turn their teams into winners (after accounting for the extra revenue winning could bring in)?
If you look at, e.g., Forbes magazine's estimate of how much teams typically make in operating profit, it's fair to say that many team owners already sacrifice some financial return in order to provide a good product for their city's fans. That is, if they took the wealth they have tied up in their teams and invested instead in stocks and bonds, they'd make more money. As to how fair it is for fans to expect them to do more in that regard, that's a question for a moral philosopher.
13) The Orioles have said in the past that their relative lack of investment into international free agents (prospects, mostly) was due to a study they had done showing a poor return on investment in that area. Are you familiar with this work, and would you agree with the above conclusion?
I'm not familiar with such studies, but I can believe that the return on int'l FAs is lower than that in the amateur draft because clubs don't have to bid competitively for players they draft. Draft day is the most important day of the year for any club; average returns on draft investments are very high. But that still leaves room for lower-but-still adequate returns in the int'l market.
14) Are prospects in general (already in pro-ball) overvalued at this point? Teams used to not guard their prospects very much in comparison to major league players, but has the pendulum swung too far to the other side?
Hard to say. The pendulum might be swinging to-and-fro this off-season; there have been trades that some observers have judged as over-valuing prospects, and others under-valuing them. In any given year, this might be function of how many teams are going "all in," and how many are backing off for a future window of opportunity.
15) The failure rate for prospects - especially pitching prospects - is higher than fans sometimes acknowledge. How sustained of a run of failure of turning prospects into major league players can realistically be looked at (by said fans) as bad luck before it becomes more likely that there are developmental issues?
Good question. Evaluating those who develop players, as well as the players themselves, might be the next great frontier in performance analysis. Figuring out how to identify the best scouts or minor league coaches, etc., is where aspiring young sabermetricians should devote considerable energy in coming years.
16) Measuring defensive value has been one of the more complicated tasks undertaken by sabermetricians. How well do you think the publicly available defensive metrics perform in general? At Camden Yards specifically?
Defensive metrics have come a long way in the last 10 years, but none are perfect; it's best to look at several and combine them with eyeball evaluations. Even when we get better metrics, though, we don't know how much of a player's performance is a result of positioning that might have been the result of his own homework or a coach's insight.
17) Are there other areas of study - for example, the prediction and prevention of injuries - which teams could research more fully to develop a competitive advantage?
Yes, better assessing injury risk is a major area of opportunity for teams.
18) How much of your work (and the work of others in the stat department) involves new advances versus fine-tuning existing ideas?
I don't have a percentage breakdown, but both are ongoing pre-occupations. Everyone I talk to who does stat analysis is full of ideas about ways to make existing tools work better, but also to develop new tools. Not too many days go by when I don't think "it'd be great to have the data to assess X" (or hear from someone thinking similarly).
19) In general, how much impact on a team's win total do you think a manager can have with (a) his on the field moves (including determining playing-time and batting order, as well is in-game decision) and (b) the behind the scenes motivation and teaching and whatnot?
Again, this is the next great area that sabermetricians should be focusing on. And it's not just "the" manager's in-game strategy, 'cause he's got a bench coach whispering in his ear, and a pitching coach who prepped up the starter that night, and a first-base coach who gave the shortstop a pep talk between innings, etc. These are "micro" events like strikeouts or sac bunts, but we haven't really made much progress seeing how they add up to wins in the same way we have quantified player productivity.
20) Is there anything you'd like to see people who analyze baseball publicly do that they currently do not (or could do better)?
As an academic, I'm a big believer in peer review. Before an article can get published in a reputable journal, two or three experts with knowledge of the specific area go over it with a fine-toothed comb. Too often in sabermetrics, we'll publish our studies before they've been properly reviewed; we trust that post-publication commenters will detect errors. We need to tighten that up a bit, or the credibility of the enterprise can suffer.
Professor Walters also passed along the following thoughts:
One key thing to remember in this business is that nobody bats 1.000 or even gets close. You do the best you can in real time, and in a distressingly large number of cases you'll do the wrong thing no matter how smart or hard-working you are.
Another thing to keep in mind is the Weaver dictum that you're never as bad as you look when you lose or as good as you look when you win. We're coming off a disappointing season, and I sense that a lot of O's fans are discouraged. But perhaps they are more discouraged than they should be: the club has a lot of strengths, and great leadership on the field and in the front office. Hang in there!
There isn't a ton of groundbreaking stuff here, but obviously Professor Walters was not in a position to divulge proprietary information (nor would he have the time to devote a blog-post worth of answers to each question given that he's a busy man and I may have taken a little too much advantage of this opportunity by asking 20 of them). I have to say that the "concerns" I had (which I saw echoed by others) when he was hired regarding whether or not he would add value to the club are pretty much answered (looks like a 'yes' to me, even if one extra person in the stat department isn't going to turn the team into a contender - my main concern regarding his application of economics to baseball has certainly been assuaged).
Also, the response to (5) was good to hear (with the same thoughts echoed in other answers). (10) and (11) were interesting in that "wise owners" are referenced as ones who will make the more prudent seeming moves. This, of course, raises the question of where the Orioles stand in that regard (as they say in text books, the answer to that will be left as an exercise to the reader). Answer (13) also seems encouraging (that study always sounded a bit fishy to me - though it still might be true, certainly).
I greatly appreciate Professor Walters taking the time to talk to me, and given that I was often more discouraged with the Orioles' decision making process than with the results on the field, I'm every so slightly heartened about the direction of the team.