403 Error on PUT https://api.smartsheet.com/2.0/webhooks/<scope> beginning July 7

ZapierIntegrations

Zapier users are trying to setup Webhooks with Smartsheet and receiving HTTP 403 on this call

PUT https://api.smartsheet.com/2.0/webhooks/<scope> (to enable the webhook)

Request data is {"enabled":true}

Example response data is { "errorCode" : 1004, "message" : "You are not authorized to perform this action.", "refId" : "xopb9p"}

Prior to this call the user is able to successfully:
GET https://api.smartsheet.com/2.0/webhooks (to list webhooks)
POST https://api.smartsheet.com/2.0/webhooks (to create the webhook)

All these calls require the same ADMIN_WEBHOOKS OAuth scope, which we have enabled.

The errors only began on July 7, and now 80% of these PUT calls are failing with this error.

Please reach out via email if any sensitive details are required, otherwise I will monitor this thread.
Thank you.

Georgie

Hi @ZapierIntegrations ,

Please note that Webhooks are automatically disabled on sheets that exceed 20,000 rows, 400 columns, or 500,000 cells - it might be that the sheet you’re monitoring is exceeding one of these values and that’s why you’re being presented with an error message. Additionally, attempting to re-enable a disabled webhook will return an error response - take a look at the API documentation here for more information. 

If the above possible causes are not the cause of the issue in your case, I’d recommend creating a Support ticket to have the team investigate this further. 

Thanks,

Georgie

ZapierIntegrations

Thanks Georgie.

I don't believe the sheet size is causing the issue, because of how widespread the issue is, and the fact that some users are seeing success when they create a new authentication with us (redo-ing the OAuth flow), so it works on the same sheet they tried (and failed) on previously.

What I don't understand is why the users can make the GET https://api.smartsheet.com/2.0/webhooks call with their auth (which we added as a check to verify the Authorization token has the right OAuth scope), but be 403 on the PUT? It seems like some change on July 7 started this, as before that, users had no issues with this process in our logs.

For now we're trying to instruct users to redo their authentication and seeing if that works for most users. If so, we will just continue with that workaround, barring any additional insight.

These are not re-enable attempts as our system is setup to create a new hook every time.

I also cannot create a support ticket as Zapier itself is not a paying customer of Smartsheet, though many of our users (over 5,000 of them and counting) likely are! It would be really helpful for us to have a way to get in touch besides this forum for that reason.

Thanks,
Jason

Alex Argumedo

In my research about webhooks and Zapier I found that the initial call (webhook creation and initial verification) requires a custom response from the server, and Zapier has no customized responses in the options, therefore the industry standard response of Zapier won't make it for SS endpoint and therefore the webhook doesn't get activated.

My guess is that you can activate the webhook with a different method and then once the webhook is active you can use it in Zapier, but I never got to try that

ZapierIntegrations

Just to clarify, we do provide the customized response to the "challenge" that Smartsheet sends, so webhooks do get successfully activated when users don't hit this HTTP 403 on the PUT call. Note that this only works if you use the official Smartsheet integration and triggers (not any custom ones or a self-made developer platform app, because the challenge code is hardcoded for the official app's triggers).

For now we've had some users re-authenticate within Zapier to Smartsheet again, and it works after that. I'm not sure why that would be the case, but barring hearing anything else we'll keep experimenting with that approach for the moment.