Edit: Just to add: I not telling you whether you should or shouldn be upset.

If ya had an agreement in regards to what is expected when it comes to this situation, then you do best to discuss that boundary or work on setting it if it means that much to you.