One alternative to traditional marriage is a common law marriage. While a common law marriage isn't technically a marriage since there's no recognized marriage ceremony, Domestic Partnership Agreement, or marriage license, as an interpersonal status it's still recognized in some states for heterosexual couples. You can show proof of a common law relationship with a notarized affidavit, laying out how long you and your partner have lived together, where you lived, any public announcement of common law marriage, and any previous marriages.

Cohabitation is not enough to qualify you and your partner for a common law marriage: you and your partner must behave like spouses and be qualified to enter a marriage (for example, you must both be of legal age).  The IRS recognizes common-law marriages, so you can file your taxes jointly. You and your partner can even use the same last name. However, in the US, no state recognizes same-sex common-law marriages (although some states do recognize same-sex marriages). The states that recognize heterosexual common law marriages are Alabama, Colorado, the District of Columbia, Iowa, Kansas, Montana, New Hampshire (for inheritance purposes only), Oklahoma, Rhode Island, South Carolina, Texas, and Utah. Other states recognize common law marriages that were notarized before a certain date: Georgia (pre-1997) Idaho (pre-1996) Ohio (pre-1991) Pennsylvania (pre-2005).

Finally, there's no such thing as a specific common-law divorce: common-law couples have to go through same divorce process as legally married couples.