If you decide to rescind access to your File Vault, simply delete the Partner User or Sub User from the Manage Collaborators page.
On this page, you will see each user that has access privileges to your account. To remove them, simply click on the little "x" on the upper right hand corner of the square. After confirming that you wish to delete the linkage, that person will immediately lose all access to your account.
If you wish to share your information again, you will have to re-invite them. You can do so from the same "Manage Collaborators" screen.