| Investment Owned Balance Principal Amount | $5.8B | $6.2B | $5.6B | $5.5B | $7.5B | $7.3B | $7.7B | $9.3B |
| Loans held for investment, Unpaid Principal/Notional | — | — | $11.7B | — | — | — | $10B | $8.4B |
| Loans held for investment, at fair value | $11.2B | $11.3B | $11.2B | $11B | $10.6B | $10.3B | $9.8B | $8.2B |
| Securitized debt loans held for investment at cost | $11.6B | $11.3B | $11.4B | $11.1B | $10.7B | $10.3B | $9.8B | $8.2B |
| Derivative Assets And Liabilities, Security Purchased Under Agreement To Resell, And Security Borrowed, Subject To Master Netting Arrangement, Deduction Of Financial Instrument Not Offset | — | — | $4.1B | $4.2B | $5.9B | $6.1B | $7.4B | $8.1B |
| Aggregate Notional Principal Outstanding | $2.3B | $2B | $2.2B | $3.2B | $3.6B | $4.5B | $5.3B | $7B |
| Derivative Notional Amount | $2.3B | $2B | $2.2B | $3.2B | $3.6B | $4.5B | $5.3B | $7B |
| Notional | $2.3B | $2B | $2.2B | $3.2B | $3.6B | $4.5B | $5.3B | $7B |
| Derivative Asset, Derivative Liability, Security Sold under Agreement to Repurchase, and Security Loaned, Subject to Master Netting Arrangement, after Offset | $2.7B | $3.2B | $2.8B | $3B | $4.6B | $4.9B | $6B | $7B |
| Derivative Asset. Derivative Liability, Security Sold under Agreement to Repurchase, and Security Loaned, Subject to Master Netting Arrangement, before Offset | $2.7B | $3.2B | $2.8B | $3B | $4.6B | $4.9B | $6B | $6.9B |
| Estimated Fair Value | $1.6B | $2.1B | $1.6B | $1.7B | $3.7B | $3.8B | $4.3B | $6B |
| Cost or Amortized Cost | $1.5B | $2.1B | $1.6B | $1.7B | $3.6B | $3.7B | $4.2B | $5.9B |
| Securitized debt at fair value, collateralized by loans held for investment, Unpaid Principal/Notional | — | — | $7.6B | — | — | — | $7.1B | $5.8B |
| Securitized debt at fair value, collateralized by Loans held for investment | $7.1B | $7.3B | $7B | $7.2B | $7B | $7B | $6.7B | $5.4B |
| Agency MBS, at fair value | $503.3M | $1B | $519.2M | $656.3M | $2.7B | $2.9B | $3.5B | $5.2B |
| Debt Securities, Available-for-Sale, Fair Value, Maturity, Allocated and Single Maturity Date, after Year 5 Through 10 | $529M | $557.3M | $490.3M | $982.4M | $2.7B | $3B | $3.4B | $4.7B |
| Debt Securities, Available-for-Sale, Amortized Cost, Maturity, Allocated and Single Maturity Date, after Year 5 Through 10 | $503.9M | $516.4M | $470M | $438.1M | $2.6B | $2.9B | $3.4B | $4.7B |
| Debt Securities, Available-for-Sale, Unrealized Loss Position | $829.7M | $399.2M | $785.4M | $871.6M | $828.8M | $974M | $418.2M | $3.2B |
| Less than 12 months | $529.1M | $94.7M | $507M | $591.7M | $559.1M | $777.5M | $242.6M | $2.9B |
| Derivative Asset. Derivative Liability, Security Sold under Agreement to Repurchase, and Security Loaned, Subject to Master Netting Arrangement, after Offset and Deduction | -$1.2B | -$1.3B | -$1.3B | -$1.2B | -$1.4B | -$1.3B | -$1.4B | -$1.2B |
| Security Sold under Agreement to Repurchase, Subject to Master Netting Arrangement, after Offset and Deduction | -$1.2B | -$1.3B | -$1.3B | -$1.2B | -$1.3B | -$1.3B | -$1.3B | -$1.2B |
| Debt Securities, Available-for-Sale, Fair Value, Maturity, Allocated and Single Maturity Date, after Year 10 | $793.9M | $1.4B | $879.1M | $506M | $778M | $580.6M | $505.4M | $939.1M |
| Debt Securities, Available-for-Sale, Amortized Cost, Maturity, Allocated and Single Maturity Date, after Year 10 | $734.2M | $1.3B | $825.2M | $962.3M | $712.7M | $521.3M | $440M | $890M |
| Available For Sale Debt Securities Discount | $841M | $828.8M | $815.7M | $805.4M | $891.7M | $813M | $799.8M | $788.3M |
| Non-Agency RMBS, at fair value (net of allowance for credit losses of $46 million and $43 million, respectively) | $1.1B | $1.1B | $1.1B | $1.1B | $1B | $868.8M | $817.3M | $756.1M |
| Unpaid principal balance | — | — | $0 | — | — | — | $871.8M | $684.6M |
| Payable for investments purchased | $464M | $123.9M | $454.7M | $38.5M | $387.9M | $193.4M | $3.3M | $611.5M |
| Debt Securities, Available-for-Sale, Amortized Cost, Maturity, Allocated and Single Maturity Date, after Year One Through Five | $158M | $185.9M | $219.3M | $246.6M | $233.2M | $225.4M | $373.4M | $335.2M |
| Debt Securities, Available-for-Sale, Fair Value, Maturity, Allocated and Single Maturity Date, after Year One Through Five | $131.5M | $163.2M | $192.4M | $210.4M | $197.9M | $190.4M | $330M | $291.9M |
| Fair value of securities sold under agreement to repurchase | — | — | $319.5M | — | — | — | $298.7M | $291.8M |
| Total Premium | $204.9M | $203.9M | $204.5M | $205.2M | $205.1M | $214.6M | $224M | $251.8M |
| Share Repurchase Program, Remaining Authorized, Amount | $217M | $250M | $250M | $250M | $250M | $250M | $250M | $250M |
| 12 months or more | $300.6M | $304.5M | $278.4M | $279.9M | $269.6M | $196.5M | $175.5M | $219.1M |
| Gross Unrealized Gains | $211.8M | $225.1M | $200.4M | $199M | $212.7M | $224.6M | $235.4M | $200.1M |
| Commitment to purchase loans | — | — | — | — | — | — | — | $187M |
| Common stock, shares authorized (in shares) | $500M | $166.7M | $166.7M | $166.7M | $166.7M | $166.7M | $166.7M | $166.7M |
| Gross Unrealized Gain Included in Accumulated Other Comprehensive Income | $188.4M | $195.4M | $174.4M | $171.6M | $165.5M | $159.4M | $153.1M | $141.8M |
| Debt Securities, Available-for-Sale, Accumulated Gross Unrealized Loss, before Tax | $140M | $115.1M | $142.1M | $129.9M | $121.9M | $111.7M | $108.6M | $117.2M |
| Debt Securities, Available-for-Sale, Unrealized Loss Position, Accumulated Loss | $140M | $115.1M | $142.1M | $129.9M | $121.9M | $111.7M | $108.6M | $117.2M |
| Indefinite- lived intangible impairment | — | — | — | — | — | — | — | $114M |
| Available For Sale Securities Gross Unrealized Losses In Deficit | $125.7M | $103.4M | $127.1M | $116.1M | $108.8M | $101.7M | $101.8M | $113.1M |
| Debt Securities, Available-for-Sale, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss | $135.4M | $114.1M | $135M | $124.5M | $117.6M | $108.1M | $106.8M | $103.4M |
| Preferred Stock, Shares Authorized | $100M | $100M | $100M | $100M | $100M | $100M | $100M | $100M |
| Common stock, shares issued (in shares) | $241M | $80.9M | $80.9M | $81M | $81M | $81.1M | $83.4M | $83.6M |
| Common Stock, Shares, Outstanding | $241M | $80.9M | $80.9M | $81M | $81M | $81.1M | $83.4M | $83.6M |
| Securitized debt, collateralized by Non-Agency RMBS | $72.1M | $72.2M | $71.2M | $70M | $68.3M | $67.6M | $66.6M | $65M |
| Gross Unrealized Gain Included in Cumulative Earnings | $23.4M | $29.8M | $26M | $27.4M | $47.2M | $65.2M | $82.3M | $58.3M |
| Derivative Asset, Derivative Liability, Security Sold under Agreement to Repurchase, and Security Loaned, Subject to Master Netting Arrangement, Collateral, Right to Reclaim Cash Not Offset | -$13.5M | -$8.7M | -$4.6M | $5.5M | -$57.6M | -$58.4M | -$61.8M | -$57.6M |
| Repurchase agreements, Gross amounts not offset in the balance sheet, Cash collateral pledged | $9.2M | $13.4M | $17.3M | $10.6M | $693K | $17.3M | $17M | $49.7M |
| Available-for-sale securities, allowance for credit loss | $23.6M | $23.9M | $28.4M | $31.8M | $36.2M | $38.3M | $43.2M | $46.1M |
| Interests in MSR financing receivables, Unpaid Principal/Notional | — | — | $0 | — | — | — | $40.9M | $37.6M |
| Debt Securities, Available-for-Sale, Amortized Cost, Maturity, Allocated and Single Maturity Date, Year One | $111M | $32M | $38.9M | $31.9M | $69.9M | $29M | $25.7M | $25.8M |
| Net unrealized gains | — | — | — | — | — | — | $24.3M | $16M |
| Debt Securities, Available-for-Sale, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss | $4.6M | $1.1M | $7M | $5.4M | $4.3M | $3.5M | $1.8M | $13.9M |
| Derivative Asset, Derivative Liability, Security Sold under Agreement to Repurchase, and Security Loaned, Subject to Master Netting Arrangement, Asset Offset | $16.6M | $6.9M | $7.1M | $8.5M | -$3.1M | -$1.5M | $4.1M | $13.5M |
| Derivative Asset, Subject to Master Netting Arrangement, Collateral, Obligation to Return Cash, Offset Against Derivative Asset | — | — | $7.1M | — | — | — | $4.1M | $13.5M |
| Debt Securities, Available-for-Sale, Fair Value, Maturity, Allocated and Single Maturity Date, Year One | $100.9M | $24.3M | $21.6M | $17.4M | $55.2M | $14.8M | $9.2M | $5.8M |
| Loan Repurchase Reserve | — | — | $0 | — | — | — | $5.8M | $5.4M |
| Available For Sale Securities, Gross Unrealized Losses Recognized In Accumulated Other Comprehensive Income | $14.3M | $11.7M | $14.9M | $13.8M | $13.1M | $10M | $6.8M | $4.1M |
| Finite-Lived Intangible Assets, Accumulated Amortization | — | — | — | — | — | — | — | $4M |
| Incentive stock program, shares reserved for future grants (in shares) | $6M | — | $5M | $6M | $4M | $4M | $4M | $3M |
| Components | 247.00 | 228.00 | 246.00 | 237.00 | 233.00 | 227.00 | 208.00 | 311.00 |
| Debt Securities, Available-for-Sale, Unrealized Loss Position, Number of Positions | $247 | $228 | $246 | $237 | $233 | $227 | $208 | $311 |
| Number of Securities, Greater than 12 months | $206 | $201 | $201 | $201 | $200 | $182 | $181 | $182 |
| Number of warehouse financing facilities under master repurchase agreements | — | — | — | — | — | — | 7.00 | 7.00 |
| Number of deferred compensation plans | — | — | 2.00 | 2.00 | 2.00 | 2.00 | 2.00 | 2.00 |
| Securities sold under agreements to repurchase, haircut | 7.5% | 7.5% | 7.5% | 7.5% | 7.5% | 7.5% | 7.5% | 7.5% |
| Common stock, par value (in dollars per share) | $0 | $0 | $0 | $0 | $0 | $0 | $0 | $0 |
| Preferred Stock, Par or Stated Value Per Share | $0 | $0 | $0 | $0 | $0 | $0 | $0 | $0 |
| Tenor spread adjustment | 0.3% | 0.3% | 0.3% | 0.3% | 0.3% | 0.3% | 0.3% | 0.3% |
| Gross Assets/Liabilities | $0 | $0 | $0 | $240K | $301K | $5.4M | $1.8M | $0 |
| Business Combination, Contingent Consideration, Liability | — | — | $15.8M | — | — | — | $2.1M | — |
| Deferred Tax Liabilities, Gross | — | — | $0 | — | — | — | $25.1M | — |
| Disallowed business interest expense carryforward | — | — | $3.6M | — | — | — | $4.6M | — |
| Deferred Tax Assets, Gross | — | — | $110.7M | — | — | — | $104.6M | — |
| Lease Liability | — | — | $0 | — | — | — | $293K | — |
| Deferred Tax Assets, Net of Valuation Allowance | — | — | $0 | — | — | — | $2.4M | — |
| Deferred Tax Assets, Operating Loss Carryforwards | — | — | $107.1M | — | — | — | $97.3M | — |
| Fixed assets | — | — | $0 | — | — | — | $109K | — |
| Deferred Tax Assets State Taxes | — | — | $0 | — | — | — | $245K | — |
| Reserve for uncertain tax position | — | — | $0 | — | — | — | $1.5M | — |
| Accrued expenses | — | — | $0 | — | — | — | $336K | — |
| Deferred Tax Assets, Valuation Allowance | — | — | $110.7M | — | — | — | $102.2M | — |
| Deferred Tax Liabilities, Net | — | — | $0 | — | — | — | $22.7M | — |
| Federal and state intangible assets | — | — | $0 | — | — | — | $24.8M | — |
| Derivative Liability | $0 | $0 | $0 | $240K | $301K | $5.4M | $1.8M | $0 |
| Derivative Liability, Subject to Master Netting Arrangement, Collateral, Right to Reclaim Cash Offset | — | — | $0 | — | — | — | $0 | $0 |
| Loans 30 To 89 Days Delinquent | — | — | $997.1M | — | — | — | $919.6M | — |
| Loans 90 or more days delinquent | — | — | $728.2M | — | — | — | $645.2M | — |
| Number of loans | — | — | 103571.00 | — | — | — | 93956.00 | — |