Google KMS on AppEngine Dev Server - Registration Clutter
This is a known issue: https://issuetracker.google.com/issues/63253097
In addition to my question on how Google KMS works with App Engine , I'm opening a new question about a related but problematic side effect.
Namely, the discovery of the Google Cloud API seems to want to take several paths outside of the AppEngine sandbox. This is not a problem in itself, but it makes testing and debugging very problematic as each attempt is logged, producing hundreds of lines such as:
INFO 03 Jul 2017 14:44:51 Sandbox denied access to file "/ Users"
INFO 03 Jul 2017 14:44:51 If it is a static file, check that
application_readable: true
app.yaml is installed in your applicationINFO 03 Jul 2017 14:44:52 Sandbox denied access to file "/ usr / lib / Acrobat9 / Resource / CMap"
INFO 03 Jul 2017 14:44:52 If it is a static file, check that
application_readable: true
app.yaml is installed in your applicationINFO 03 Jul 2017 14:44:52 Sandbox denied access to the file "/ usr / lib / Acrobat8 / Resource / CMap"
INFO 03 Jul 2017 14:44:52 If it is a static file, check that
application_readable: true
app.yaml is installed in your application
Is there a way to turn off or turn off these queries or logs?
EDIT * One of the many results printed by adding traceback.print_stack()
to log_access_check_fail
in stub.py
(sorry, this is not very readable):
Test # 1: Sentry.io Raven Imports pkg_resources
File "/Users/bmh/testapp/application.py", line 3, in <module>
from nassau.application import app, sentry
File "/Users/bmh/testapp/nassau/application.py", line 28, in <module>
from raven.contrib.flask import Sentry
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/raven/__init__.py", line 16, in <module>
VERSION = __import__('pkg_resources') \
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3037, in <module>
@_call_aside
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3021, in _call_aside
f(*args, **kwargs)
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3050, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 646, in _build_master
ws = cls()
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 639, in __init__
self.add_entry(entry)
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 695, in add_entry
for dist in find_distributions(entry, True):
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2006, in find_on_path
path_item = _normalize_cached(path_item)
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2217, in _normalize_cached
_cache[filename] = result = normalize_path(filename)
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2210, in normalize_path
return os.path.normcase(os.path.realpath(filename))
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/posixpath.py", line 375, in realpath
path, ok = _joinrealpath('', filename, {})
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/posixpath.py", line 400, in _joinrealpath
if not islink(newpath):
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/posixpath.py", line 135, in islink
st = os.lstat(path)
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/stubs.py", line 286, in __call__
log_access_check_fail(path)
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/stubs.py", line 51, in log_access_check_fail
traceback.print_stack()
I removed the dependency on Sentry / Raven, but the stack traces continue using:
Test # 2: Importing Flasks pkgutil
File "/Users/bmh/testapp/application.py", line 3, in <module>
from nassau.application import app
File "/Users/bmh/testapp/nassau/application.py", line 72, in <module>
app = Flask('nassau')
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/flask/app.py", line 331, in __init__
instance_path = self.auto_find_instance_path()
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/flask/app.py", line 622, in auto_find_instance_path
prefix, package_path = find_package(self.import_name)
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/site-packages/flask/helpers.py", line 661, in find_package
loader = pkgutil.get_loader(root_mod_name)
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py", line 475, in find_loader
loader = importer.find_module(fullname)
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py", line 184, in find_module
path = [os.path.realpath(self.path)]
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/posixpath.py", line 375, in realpath
path, ok = _joinrealpath('', filename, {})
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/posixpath.py", line 400, in _joinrealpath
if not islink(newpath):
File "/Users/bmh/testapp/python-virtualenv/lib/python2.7/posixpath.py", line 135, in islink
st = os.lstat(path)
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/stubs.py", line 286, in __call__
log_access_check_fail(path)
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/stubs.py", line 51, in log_access_check_fail
traceback.print_stack()
There must be a better way to represent the stack trace on a stack overflow :)
source to share
I work for dev_appserver as a full time engineer. This log message is reported to / google / appengine / tools / devappserver 2 / python / stubs.py in the methodlog_access_check_fail
We added this log message to remind users of the dev_appserver file blocking behavior.
Recommended workaround: warning dev_appserver.py --log_level
- log_level sets the log level in the local python execution process. Log_access_check_fail () is set to INFO and will not be logged. NOTE. This flag will also prevent INFO level logging in your application.
In the meantime, we are working on fixing this log_access_check_fail method. This can reduce the log_access_check_fail () logging level to DEBUG.
source to share
This is pretty annoying and there is a bug report for it .
In the meantime, we can filter out these messages, but we need to figure out which ones logger
they use. See the docs for more information on how the logging entries are handled. Taking a look at the source for stubs.py
, I found:
logging.info('Sandbox prevented access to file "%s"', filename)
logging.info('If it is a static file, check that '
'`application_readable: true` is set in your app.yaml')
So they use a root registrar (usually bad practice). To filter these messages in the root log add the following to appengine_config.py
:
import logging
class StubsFilter(logging.Filter):
def filter(self, record):
return 'stubs.py' != record.filename
logging.root.addFilter(StubsFilter())
source to share
To me, these messages seem to come from a GAE (.local) file called stubs.py.
-
In my first case, stubs.py was looking for files that weren't there. So you can check if "/ usr / lib / Acrobat8 / Resource / CMap" actually exists. Then, starting with stubs.py, try to find which file is trying to load it .
-
My project is using web.py, I started getting messages (stubs.py) again, this time about .pyc files. I put skip_files: at the bottom of my app.yaml file and it suppresses messages, but I haven't tested them if my app still fully compiles.
skip_files: - ^(.*/)?\.pyc$
-
Learn more about the Python GAE runtime and sandbox here. https://cloud.google.com/appengine/docs/standard/python/runtime
source to share