Only thing I can think of is common law marriage. After a certain point, it
becomes understood that they are an item. In that sense it is a truth.
As you implied, there is a status to marriage, mjp, where shacking up carries a
bit of shame. I believe he respected Jane so much that he desired to lend a bit
of credence to their relationship in the eyes of the FBI, and for Jane's sake.
Albeit a stretch, it's not too far a stretch, and it has a bit of the truth in it.
Confronting power with the truth is the game.
Just a guess. I may be way off base, but Power intimidates, and it may have
been the kind of bold move CB wanted, to be able to look those in power in
the eye and smirk, while honoring that, and those, whom he loved.