Limit the availability of panels in the Views interface

I am trying to customize the view interface so that users can only see certain panels (i.e. only fields, filters, and style preference panels). I've tested SimpleViews and UI Views: Edit Basic Module Settings , but it wasn't that useful.

Hiding panels with jQuery wasn't a good way either.

Anyone have a solution?