Morningstar
Yesterday the U.S. Securities and Exchange Commission sued Morningstar Credit Ratings LLC for being too generous when it rated commercial mortgage-backed securities, and it is another software-design horror story, though of an opposite kind.
In commercial mortgage-backed securities, CMBS, banks package a bunch of commercial mortgages into a pool and then issue securities—“certificates”—backed by the pool. If mortgages in the pool default, some of the certificates—the junior tranches—will lose money first, while other certificates—the senior tranches—won’t lose anything until the junior tranches are completely wiped out. This means that the senior tranches are very safe and can get AAA ratings from ratings firms like Morningstar, which means very conservative and regulated investors can buy them.
Much of the game of issuing CMBS, then, is about maximizing the portion of the securities that get high (ideally AAA) ratings. If you pool a bunch of conservative low-loan-to-value loans against a diversified pool of good properties into a CMBS, a lot of the certificates will be AAA; if you pool a bunch of risky loans against garbage, more of the certificates will be BBB or worse.
Ratings firms have a natural and well understood conflict of interest. On the one hand, they should try to give things the correct ratings; if you hand them a pool of risky loans against garbage, they should give the securities relatively low ratings (not much AAA), to reflect the actual probability of default and uphold their standards of intellectual honesty and protect innocent investors and so forth. On the other hand, the banks that issue CMBS want good ratings (lots of AAA), and if a ratings firm doesn’t give out lots of AAA ratings the banks (who build the CMBS, pick the ratings agencies and pay them) will go somewhere else. Also to be fair investors want lots of AAA ratings; most (not all!) of the time, everyone is happiest if they just all pretend that everything is AAA and the ratings agencies back them up.
But this conflict is, again, very well understood, and politicians and regulators hate it, and the SEC monitors the ratings firms to make sure they’re not just giving everything AAA ratings. One way to do that is by regulating their ratings models, or at least their disclosure of their models. The ratings firms have to have quantitative models that take inputs—about the cash flows of the buildings in the CMBS pool, etc.—and apply some predictable process to them to decide how much of the pool gets rated AAA, etc. And they have to disclose how those models work.
In theory the purpose of this is so that investors can understand what the ratings mean. In practice the purpose is to let the SEC sue ratings firms if they nudge ratings up on an ad hoc basis. The investors don’t actually care how the model works or what the ratings mean; they just want to buy AAA-rated stuff with high yields. But once the ratings firm writes down how its model works, if it deviates from the model to please a bank, the SEC can sue it. “You said that you rated CMBS in a principled way based on standard criteria, and instead you just rated everything AAA, so, fraud.” From the SEC’s press release yesterday:
“To increase transparency and guard against conflicts of interest, the federal securities laws require credit rating agencies to disclose how ratings are determined and to have effective internal controls to ensure they adhere to their ratings methodologies,” said Daniel Michael, Chief of the SEC Enforcement Division’s Complex Financial Instruments Unit. “In this action, the complaint alleges that Morningstar failed on both counts by permitting analysts to make undisclosed adjustments over which Morningstar had no effective internal controls.”
Fine. Morningstar had a model for rating CMBS, which it disclosed in documents on its website called “CMBS New-Issue Ratings Opinions” and “CMBS Subordination Model.” Here’s how the SEC describes it:
The first step of Morningstar’s rating process was to underwrite a representative sample of the pool of commercial real estate loans that collateralized each CMBS transaction. … Through this underwriting process, Morningstar calculated the expected net cash flow that each commercial property would generate over the life of the loan, along with the value of each property, using a capitalization rate that Morningstar determined. As a result, the key outputs of the underwriting process were the net cash flow and capitalization rate for each loan.
The next step of Morningstar’s disclosed methodology, as explained in the publicly available CMBS Subordination Model document, was to input each loan’s net cash flow and capitalization rate from the underwriting process into Morningstar’s Subordination Model, an Excel spreadsheet. The Subordination Model then subjected these values to “defined sets of stresses” to assess the likelihood of loans to default at each rating category. The model’s outputs showed the loans’ losses under various economic scenarios, expressed as a percentage of the total value of the CMBS certificates being issued. Those percentages were the model-generated subordination levels, or credit enhancement, for the various rating categories.
Intuitively, if severe stress would lead to 23% defaults, then at most 77% of the pool could be AAA-rated, etc.
Did you catch the bad words in that passage? The bad words were “an Excel spreadsheet.” Morningstar had a ratings model that was subject to careful regulatory scrutiny; Morningstar disclosed how it worked, and investors supposedly relied on that disclosure when they bought CMBS. But the model lived in Microsoft Excel. If you were the Morningstar analyst working on rating a new deal, you copied your last deal into New_Deal.xlsx, and you started typing. You could type lots of things! It’s Excel, why not. The SEC complains:
Morningstar failed to disclose that a central feature of its Subordination Model allowed analysts to make “loan-specific” adjustments to the disclosed net cash flow and capitalization rate stresses.
The adjustments were made in two columns of cells in the Excel spreadsheet … labelled “LOAN SPECIFIC ADJUSTMENTS TO BASE N[ET] C[ASH] F[LOW] STRESS,” and the other was labelled “LOAN SPECIFIC ADJUSTMENTS TO BASE CAP RATE STRESS.” …
Other than the labels, Morningstar provided its analysts with no criteria or guidance for when or how to employ these adjustments. … Nor did the Subordination Model constrain how large the analyst-employed stress adjustments could be.
Even the column labels in the Subordination Model’s Excel spreadsheet failed to constrain the use of the adjustments. Morningstar’s corporate representative said that analysts could use the adjustments for reasons having nothing to do with a specific loan, such as to nudge a rating produced by the model to align with expectations. Specifically, Morningstar’s analysts could use the “loan-specific” stress adjustments when “the aggregate levels that are spit out by the model are either too high or too low relative to other similar transactions we’ve looked at.”
“Even the column labels in the Subordination Model’s Excel spreadsheet failed to constrain the use of the adjustments”! Even the column labels! The analysts looked at the columns labeled “do some fudging here, but not in a bad way,” and they did some fudging in a bad way, and now the SEC is mad. Even the column labels! Citi’s problem is that it had an opaque high-strung bit of software that, if you didn’t check exactly the right boxes, would wire hundreds of millions of dollars out of the bank for its own perverse amusement. Morningstar’s problem is that it put a highly regulated model in a regular old Excel spreadsheet where analysts could type whatever they wanted, and did.
I wonder how many Highly Regulated Excel Spreadsheets there are in the financial industry. Thousands, surely. There you are, doing your job, in your Highly Regulated Excel Spreadsheet. And you get some result you don’t like and you say, well, I dunno, I’ll just multiply everything by 1.02, that seems fine. And then years later regulators are like, no no no, that was a Highly Regulated Excel Spreadsheet, the column labels were sacrosanct, you can’t just type whatever you want there. But of course you could just type whatever you wanted there, because it was in Excel and that’s how Excel works.
Hmmm.