You will need:
Create a HubSpot workflow to process the SendGrid events. The trigger should look like the example below:
This will enrol contacts who have opted out of these groups on the SendGrid side.
The next step is to set the correct status on the HubSpot end. Select the action "Manage email subscription status" and then select the matching HubSpot subscription group.
Repeat this for every subscription group you have.
Example: