Can't write more than 204800 characters to the file?
EDIT 2: For those of you who would like to take a closer look at the code, here it is: https://github.com/pikzen/ffbookmark/blob/python-rewrite/ffbookmark.py
I'm having a bit of a problem here: python throws an IOError whenever I try to write more than 204800 characters to a file. I tried on another computer and it crashed at 768k chars. Is this a python issue, is the OS anything limited? Here is the code I'm using:
with open('out.json', 'w') as f:
json.dump(items, f)
items
- a simple dictionary. I am creating it from an HTML file containing about 800 elements. Each element is structured as follows:
bookmark = {}
bookmark["title"] = link.contents[0]
bookmark["id"] = id
bookmark["parent"] = 5
bookmark["dateAdded"] = 1
bookmark["lastModified"] = 1
bookmark["type"] = "text/x-moz-place"
uri = link.get('href')
# Shaarli self links are totally messed up : ?xGRpkrp
# But we can't simply oust links containing '?'s, because
# php uses it, and pretty much everything does
# however, if there not dot, we can assume it a
# Shaarli link.
# If it not, well too bad, false positive.
if "?" in uri and not '.' in uri:
bookmark['uri'] = "about:blank"
else:
bookmark['uri'] = uri
id += 1
try:
# This line messes up when the end of the file has been reached
# Rather than coding properly, let just catch the exception
desc = link.parent.next_sibling.next_sibling
if desc and desc.name == "dd":
bookmark["annos"] = []
annos = {}
annos["name"] = "bookmarkProperties/description"
annos["flags"] = 0
annos["expires"] = 4
annos["mimeType"] = ""
annos["type"] = 3
annos["value"] = desc.contents[0]
bookmark["annos"].append(annos)
Output:
IOError (Errno 27) : File too large
EDIT: More info: Python info:
$ python --version
Python 2.7.3
Used OS:
Linux Mint 13: limit: 204.8kB
Debian 6.0: limit: 768kB
source to share
This is not a Python bug, but a limitation of the filesystem you are writing to, or an equivalent (artificial) limitation of your operating system.
Use ulimit -f
to check the size of your file. He has to say unlimited
. If not, you will most likely want to edit /etc/security/limits.conf
. You can use the following command to find the problematic configuration:
grep fsize /etc/security/limits.conf /etc/security/limits.d/ -r
You can also check the file systemquota
or grsecurity limit.
Finally, you can check mount
to make sure the file system you plan to install is what you see.
source to share