Skip to content

/python-twitter

Browse files

updated setup.py to be wheel compatible, changed README to rst, remov… 

…ed requirements.txt and other config and copyright cleanups
1 parent ba9794e commit 1a28a85c334a9c158db68af2b14580e13aa83b47 @bear committed Jul 10, 2014
Unified Split
Showing with 241 additions and 3,749 deletions.
  1. +36 −0 AUTHORS.rst
  2. +6 −0 CHANGES
  3. +1 −1 COPYING
  4. +1 −4 MANIFEST.in
  5. +0 −212 README.md
  6. +155 −0 README.rst
  7. +4 −4 python-twitter.spec
  8. +0 −5 requirements.txt
  9. +38 −58 setup.py
  10. +0 −316 simplejson/__init__.py
  11. +0 −2,265 simplejson/_speedups.c
  12. +0 −348 simplejson/decoder.py
  13. +0 −436 simplejson/encoder.py
  14. +0 −65 simplejson/scanner.py
  15. +0 −35 simplejson/tool.py
View
36 AUTHORS.rst
@@ -0,0 +1,36 @@
+Originally two libraries by DeWitt Clinton and Mike Taylor which was then merged into python-twitter.
+
+Now it's a full-on open source project with many contributors over time:
+
+* Jodok Batlogg,
+* Kyle Bock,
+* Brad Choate,
+* Robert Clarke,
+* Jim Cortez,
+* Pierre-Jean Coudert,
+* Aish Raj Dahal,
+* Thomas Dyson,
+* Jim Easterbrook
+* Yoshinori Fukushima,
+* Hameedullah Khan,
+* Osama Khalid,
+* Omar Kilani,
+* Domen Kožar,
+* Robert Laquey,
+* Jason Lemoine,
+* Pradeep Nayak,
+* Ian Ozsvald,
+* Nicolas Perriault,
+* Glen Tregoning,
+* Lars Weiler,
+* Sebastian Wiesinger,
+* Jake Robinson,
+* abloch,
+* cahlan,
+* dpslwk,
+* edleaf,
+* ecesena,
+* git-matrix,
+* sbywater,
+* thefinn93,
+* themylogin,
View
6 CHANGES
@@ -1,3 +1,9 @@
+2014-07-10
+ update copyright years
+ change setup.py to allow installing via wheel
+ renamed README.md to README.rst
+ added AUTHORS.rst
+
2014-02-17
changed version to 1.3 and then to 1.3.1 because I forgot to include CHANGES
fix Issue 143 - GetStatusOembed() url parameter was being stomped on
View
2 COPYING
@@ -1,4 +1,4 @@
- Copyright 2007-2013 The Python-Twitter Developers
+ Copyright 2007-2014 The Python-Twitter Developers
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
View
5 MANIFEST.in
@@ -2,8 +2,5 @@ include CHANGES
include COPYING
include LICENSE
include NOTICE
-include README.md
-include *.py
-recursive-include examples *.py
-recursive-include testdata *.json
+include *.rst
prune .DS_Store
View
212 README.md
@@ -1,212 +0,0 @@
-# Python Twitter
-
-**A Python wrapper around the Twitter API.**
-
-Author: The Python-Twitter Developers <[email protected]>
-
-## Introduction
-
-This library provides a pure Python interface for the [Twitter API](https://dev.twitter.com/). It works with Python versions from 2.5 to 2.7. Python 3 support is under development.
-
-[Twitter](http://twitter.com) provides a service that allows people to connect via the web, IM, and SMS. Twitter exposes a [web services API](http://dev.twitter.com/doc) and this library is intended to make it even easier for Python programmers to use.
-
-## Building
-
-You can install python twitter from `pip` :
-
- $ pip install python-twitter
-
-From source:
-
-Install the dependencies:
-
-- [Requests](http://docs.python-requests.org/en/latest/)
-- [SimpleJson](http://cheeseshop.python.org/pypi/simplejson)
-- [Requests OAuthlib](https://requests-oauthlib.readthedocs.org/en/latest/)
-
-Alternatively use `pip`:
-
- $ pip install -r requirements.txt
-
-Download the latest `python-twitter` library from: http://code.google.com/p/python-twitter/
-
-Extract the source distribution and run:
-
-```
-$ python setup.py build
-$ python setup.py install
-```
-
-*Testing*
-
-With setuptools installed:
-
-```
-$ python setup.py test
-```
-
-Without setuptools installed:
-
-```
-$ python twitter_test.py
-```
-
-## Getting the code
-
-The code is hosted at [Github](https://github.com/bear/python-twitter).
-
-Check out the latest development version anonymously with:
-
-```
- $ git clone git://github.com/bear/python-twitter.git
- $ cd python-twitter
-```
-
-## Documentation
-
-View the last release API documentation at: [http://dev.twitter.com/doc](http://dev.twitter.com/doc)
-
-## Using
-
-The library provides a Python wrapper around the Twitter API and the Twitter data model.
-
-**Model:**
-
-The three model classes are `twitter.Status`, `twitter.User`, and `twitter.DirectMessage`. The API methods return instances of these classes.
-
-To read the full API for `twitter.Status`, `twitter.User`, or `twitter.DirectMessage`, run:
-
-```
-$ pydoc twitter.Status
-$ pydoc twitter.User
-$ pydoc twitter.DirectMessage
-```
-
-*API:*
-
-The API is exposed via the `twitter.Api` class.
-
-The python-twitter library now only supports oAuth authentication as the Twitter devs have indicated that OAuth is the only method that will be supported moving forward.
-
-To create an instance of the `twitter.Api` with login credentials (Twitter now requires an oAuth Access Token for all API calls)
-
-```
->>> import twitter
->>> api = twitter.Api(consumer_key='consumer_key',
- consumer_secret='consumer_secret',
- access_token_key='access_token',
- access_token_secret='access_token_secret')
-```
-
-To see if your credentials are successful:
-
-```
->>> print api.VerifyCredentials()
-{"id": 16133, "location": "Philadelphia", "name": "bear"}
-```
-
-**NOTE -** much more than the small sample given here will print
-
-To fetch a single user's public status messages, where `user` is either
-a Twitter *short name* or their user id.
-
-```
->>> statuses = api.GetUserTimeline(user)
->>> print [s.text for s in statuses]
-```
-
-To fetch a list a user's friends (requires authentication):
-
-```
->>> users = api.GetFriends()
->>> print [u.name for u in users]
-```
-
-To post a Twitter status message (requires authentication):
-
-```
->>> status = api.PostUpdate('I love python-twitter!')
->>> print status.text
-I love python-twitter!
-```
-
-There are many more API methods, to read the full API documentation:
-
-```
-$ pydoc twitter.Api
-```
-
-## Todo
-
-Patches and bug reports are [welcome](https://github.com/bear/python-twitter/issues/new), just please keep the style consistent with the original source.
-
-Add more example scripts.
-
-The twitter.Status and `twitter.User` classes are going to be hard to keep in sync with the API if the API changes. More of the code could probably be written with introspection.
-
-Statement coverage of `twitter_test` is only about 80% of twitter.py.
-
-The `twitter.Status` and `twitter.User` classes could perform more validation on the property setters.
-
-## More Information
-
-Please visit [the google group](http://groups.google.com/group/python-twitter) for more discussion.
-
-## Contributors
-
-Originally two libraries by DeWitt Clinton and Mike Taylor which was then merged into python-twitter.
-
-Now it's a full-on open source project with many contributors over time:
-
-* Jodok Batlogg,
-* Kyle Bock,
-* Brad Choate,
-* Robert Clarke,
-* Jim Cortez,
-* Pierre-Jean Coudert,
-* Aish Raj Dahal,
-* Thomas Dyson,
-* Jim Easterbrook
-* Yoshinori Fukushima,
-* Hameedullah Khan,
-* Osama Khalid,
-* Omar Kilani,
-* Domen Kožar,
-* Robert Laquey,
-* Jason Lemoine,
-* Pradeep Nayak,
-* Ian Ozsvald,
-* Nicolas Perriault,
-* Glen Tregoning,
-* Lars Weiler,
-* Sebastian Wiesinger,
-* Jake Robinson,
-* abloch,
-* cahlan,
-* dpslwk,
-* edleaf,
-* ecesena,
-* git-matrix,
-* sbywater,
-* thefinn93,
-* themylogin,
-
-and the rest of the python-twitter mailing list.
-
-## License
-
-```
-Copyright 2007-2013 The Python-Twitter Developers
-
-Licensed under the Apache License, Version 2.0 (the 'License');
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an 'AS IS' BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-```
View
155 README.rst
@@ -0,0 +1,155 @@
+Python Twitter
+
+A Python wrapper around the Twitter API.
+
+By the Python-Twitter Developers <[email protected]>
+
+============
+Introduction
+============
+
+This library provides a pure Python interface for the `Twitter API https://dev.twitter.com/`. It works with Python versions from 2.5 to 2.7. Python 3 support is under development.
+
+`Twitter http://twitter.com` provides a service that allows people to connect via the web, IM, and SMS. Twitter exposes a `web services API http://dev.twitter.com/doc` and this library is intended to make it even easier for Python programmers to use.
+
+==========
+Installing
+==========
+
+You can install python-twitter using::
+
+ $ pip install python-twitter
+
+Testing::
+
+ $ python twitter_test.py
+
+================
+Getting the code
+================
+
+The code is hosted at https://github.com/bear/python-twitter
+
+Check out the latest development version anonymously with::
+
+ $ git clone git://github.com/bear/python-twitter.git
+ $ cd python-twitter
+
+Dependencies
+
+* [Requests](http://docs.python-requests.org/en/latest/)
+* [SimpleJson](http://cheeseshop.python.org/pypi/simplejson)
+* [Requests OAuthlib](https://requests-oauthlib.readthedocs.org/en/latest/)
+
+=============
+Documentation
+=============
+
+View the last release API documentation at: http://dev.twitter.com/doc
+
+=====
+Using
+=====
+
+The library provides a Python wrapper around the Twitter API and the Twitter data model.
+
+-----
+Model
+-----
+
+The three model classes are ``twitter.Status``, `twitter.User``, and ``twitter.DirectMessage``. The API methods return instances of these classes.
+
+To read the full API for ``twitter.Status``, ``twitter.User``, or ``twitter.DirectMessage``, run::
+
+ $ pydoc twitter.Status
+ $ pydoc twitter.User
+ $ pydoc twitter.DirectMessage
+
+---
+API
+---
+
+The API is exposed via the ``twitter.Api`` class.
+
+The python-twitter library now only supports oAuth authentication as the Twitter devs have indicated that OAuth is the only method that will be supported moving forward.
+
+To create an instance of the ``twitter.Api`` with login credentials (Twitter now requires an oAuth Access Token for all API calls)::
+
+ >>> import twitter
+ >>> api = twitter.Api(consumer_key='consumer_key',
+ consumer_secret='consumer_secret',
+ access_token_key='access_token',
+ access_token_secret='access_token_secret')
+
+To see if your credentials are successful::
+
+ >>> print api.VerifyCredentials()
+ {"id": 16133, "location": "Philadelphia", "name": "bear"}
+
+**NOTE**: much more than the small sample given here will print
+
+To fetch a single user's public status messages, where ``user`` is either a Twitter *short name* or their user id::
+
+ >>> statuses = api.GetUserTimeline(user)
+ >>> print [s.text for s in statuses]
+
+To fetch a list a user's friends (requires authentication)::
+
+ >>> users = api.GetFriends()
+ >>> print [u.name for u in users]
+
+To post a Twitter status message (requires authentication)::
+
+ >>> status = api.PostUpdate('I love python-twitter!')
+ >>> print status.text
+ I love python-twitter!
+
+There are many more API methods, to read the full API documentation::
+
+ $ pydoc twitter.Api
+
+----
+Todo
+----
+
+Patches and bug reports are [welcome](https://github.com/bear/python-twitter/issues/new), just please keep the style consistent with the original source.
+
+Add more example scripts.
+
+The twitter.Status and ``twitter.User`` classes are going to be hard to keep in sync with the API if the API changes. More of the code could probably be written with introspection.
+
+Statement coverage of ``twitter_test`` is only about 80% of twitter.py.
+
+The ``twitter.Status`` and ``twitter.User`` classes could perform more validation on the property setters.
+
+----------------
+More Information
+----------------
+
+Please visit `the google group http://groups.google.com/group/python-twitter` for more discussion.
+
+------------
+Contributors
+------------
+
+Originally two libraries by DeWitt Clinton and Mike Taylor which was then merged into python-twitter.
+
+Now it's a full-on open source project with many contributors over time. See AUTHORS.rst for the complete list.
+
+-------
+License
+-------
+
+| Copyright 2007-2014 The Python-Twitter Developers
+|
+| Licensed under the Apache License, Version 2.0 (the 'License');
+| you may not use this file except in compliance with the License.
+| You may obtain a copy of the License at
+|
+| http://www.apache.org/licenses/LICENSE-2.0
+|
+| Unless required by applicable law or agreed to in writing, software
+| distributed under the License is distributed on an 'AS IS' BASIS,
+| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+| See the License for the specific language governing permissions and
+| limitations under the License.
View
8 python-twitter.spec
@@ -1,8 +1,8 @@
%{!?python_sitelib: %define python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")}
Name: python-twitter
-Version: 1.1
-Release: 1%{?dist}
+Version: 2.0
+Release: %{?dist}
Summary: Python Interface for Twitter API
Group: Development/Libraries
@@ -12,7 +12,7 @@ Source0: http://python-twitter.googlecode.com/files/%{name}-%{version}.ta
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildArch: noarch
-Requires: python >= 2.4, python-simplejson >= 2.0.7
+Requires: python >= 2.5, python-simplejson >= 2.0.7
BuildRequires: python-setuptools
@@ -40,7 +40,7 @@ rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
-%doc PKG-INFO README CHANGES COPYING LICENSE doc/twitter.html
+%doc README.rst CHANGES COPYING LICENSE doc/twitter.html
# For noarch packages: sitelib
%{python_sitelib}/*
View
5 requirements.txt
@@ -1,5 +0,0 @@
-requests==2.2.0
-requests-oauthlib==0.4.0
-simplejson==3.3.2
-wsgiref==0.1.2
-oauthlib==0.6.0
View
96 setup.py
@@ -1,6 +1,6 @@
#!/usr/bin/env python
#
-# Copyright 2007-2013 The Python-Twitter Developers
+# Copyright 2007-2014 The Python-Twitter Developers
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -16,61 +16,41 @@
'''The setup and build script for the python-twitter library.'''
-__author__ = '[email protected]'
-__version__ = '2.0'
-
-
-# The base package metadata to be used by both distutils and setuptools
-METADATA = dict(
- name = "python-twitter",
- version = __version__,
- packages = ['twitter'],
- author='The Python-Twitter Developers',
- author_email='[email protected]',
- description='A Python wrapper around the Twitter API',
- license='Apache License 2.0',
- url='https://github.com/bear/python-twitter',
- keywords='twitter api',
-)
-
-# Extra package metadata to be used only if setuptools is installed
-SETUPTOOLS_METADATA = dict(
- install_requires = ['setuptools', 'simplejson', 'requests', 'requests_oauthlib'],
- include_package_data = True,
- classifiers = [
- 'Development Status :: 5 - Production/Stable',
- 'Intended Audience :: Developers',
- 'License :: OSI Approved :: Apache Software License',
- 'Topic :: Software Development :: Libraries :: Python Modules',
- 'Topic :: Communications :: Chat',
- 'Topic :: Internet',
- ],
- test_suite = 'twitter_test.suite',
+import os
+
+from setuptools import setup, find_packages
+
+def read(*paths):
+ """Build a file path from *paths* and return the contents."""
+ with open(os.path.join(*paths), 'r') as f:
+ return f.read()
+
+setup(
+ name='python-twitter',
+ version='2.0',
+ author='The Python-Twitter Developers',
+ author_email='[email protected]',
+ license='Apache License 2.0',
+ url='https://github.com/bear/python-twitter',
+ keywords='twitter api',
+ description='A Python wrapper around the Twitter API',
+ long_description=(read('README.rst') + '\n\n' +
+ read('AUTHORS.rst') + '\n\n' +
+ read('CHANGES')),
+ packages=find_packages(exclude=['tests*']),
+ install_requires = ['simplejson', 'requests', 'requests-oauthlib'],
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Intended Audience :: Developers',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Operating System :: OS Independent',
+ 'Topic :: Software Development :: Libraries :: Python Modules',
+ 'Topic :: Communications :: Chat',
+ 'Topic :: Internet',
+ 'Programming Language :: Python',
+ 'Programming Language :: Python :: 2',
+ 'Programming Language :: Python :: 2.7',
+ 'Programming Language :: Python :: 3',
+ 'Programming Language :: Python :: 3.3',
+ ],
)
-
-
-def Read(file):
- return open(file).read()
-
-def BuildLongDescription():
- return '\n'.join([Read('README.md'), Read('CHANGES')])
-
-def Main():
- # Build the long_description from the README and CHANGES
- METADATA['long_description'] = BuildLongDescription()
-
- # Use setuptools if available, otherwise fallback and use distutils
- try:
- import setuptools
- METADATA.update(SETUPTOOLS_METADATA)
- setuptools.setup(**METADATA)
- except ImportError:
- import sys
- sys.stderr.write("Could not import setuptools, using distutils")
- sys.stderr.write("NOTE: You will need to install dependencies manualy")
- import distutils.core
- distutils.core.setup(**METADATA)
-
-
-if __name__ == '__main__':
- Main()
View
316 simplejson/__init__.py
@@ -1,316 +0,0 @@
-r"""JSON (JavaScript Object Notation) <http://json.org> is a subset of
-JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data
-interchange format.
-
-:mod:`simplejson` exposes an API familiar to users of the standard library
-:mod:`marshal` and :mod:`pickle` modules. It is the externally maintained
-version of the :mod:`json` library contained in Python 2.6, but maintains
-compatibility with Python 2.4 and Python 2.5 and (currently) has
-significant performance advantages, even without using the optional C
-extension for speedups.
-
-Encoding basic Python object hierarchies::
-
- >>> import simplejson as json
- >>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
- '["foo", {"bar": ["baz", null, 1.0, 2]}]'
- >>> print json.dumps("\"foo\bar")
- "\"foo\bar"
- >>> print json.dumps(u'\u1234')
- "\u1234"
- >>> print json.dumps('\\')
- "\\"
- >>> print json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True)
- {"a": 0, "b": 0, "c": 0}
- >>> from StringIO import StringIO
- >>> io = StringIO()
- >>> json.dump(['streaming API'], io)
- >>> io.getvalue()
- '["streaming API"]'
-
-Compact encoding::
-
- >>> import simplejson as json
- >>> json.dumps([1,2,3,{'4': 5, '6': 7}], separators=(',',':'))
- '[1,2,3,{"4":5,"6":7}]'
-
-Pretty printing::
-
- >>> import simplejson as json
- >>> s = json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4)
- >>> print '\n'.join([l.rstrip() for l in s.splitlines()])
- {
- "4": 5,
- "6": 7
- }
-
-Decoding JSON::
-
- >>> import simplejson as json
- >>> obj = [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]
- >>> json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]') == obj
- True
- >>> json.loads('"\\"foo\\bar"') == u'"foo\x08ar'
- True
- >>> from StringIO import StringIO
- >>> io = StringIO('["streaming API"]')
- >>> json.load(io)[0] == 'streaming API'
- True
-
-Specializing JSON object decoding::
-
- >>> import simplejson as json
- >>> def as_complex(dct):
- ... if '__complex__' in dct:
- ... return complex(dct['real'], dct['imag'])
- ... return dct
- ...
- >>> json.loads('{"__complex__": true, "real": 1, "imag": 2}',
- ... object_hook=as_complex)
- (1+2j)
- >>> import decimal
- >>> json.loads('1.1', parse_float=decimal.Decimal) == decimal.Decimal('1.1')
- True
-
-Specializing JSON object encoding::
-
- >>> import simplejson as json
- >>> def encode_complex(obj):
- ... if isinstance(obj, complex):
- ... return [obj.real, obj.imag]
- ... raise TypeError("%r is not JSON serializable" % (o,))
- ...
- >>> json.dumps(2 + 1j, default=encode_complex)
- '[2.0, 1.0]'
- >>> json.JSONEncoder(default=encode_complex).encode(2 + 1j)
- '[2.0, 1.0]'
- >>> ''.join(json.JSONEncoder(default=encode_complex).iterencode(2 + 1j))
- '[2.0, 1.0]'
-
-
-Using simplejson.tool from the shell to validate and pretty-print::
-
- $ echo '{"json":"obj"}' | python -msimplejson.tool
- {
- "json": "obj"
- }
- $ echo '{ 1.2:3.4}' | python -msimplejson.tool
- Expecting property name: line 1 column 2 (char 2)
-"""
-__version__ = '2.0.7'
-__all__ = [
- 'dump', 'dumps', 'load', 'loads',
- 'JSONDecoder', 'JSONEncoder',
-]
-
-from decoder import JSONDecoder
-from encoder import JSONEncoder
-
-_default_encoder = JSONEncoder(
- skipkeys=False,
- ensure_ascii=True,
- check_circular=True,
- allow_nan=True,
- indent=None,
- separators=None,
- encoding='utf-8',
- default=None,
-)
-
-def dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,
- allow_nan=True, cls=None, indent=None, separators=None,
- encoding='utf-8', default=None, **kw):
- """Serialize ``obj`` as a JSON formatted stream to ``fp`` (a
- ``.write()``-supporting file-like object).
-
- If ``skipkeys`` is ``True`` then ``dict`` keys that are not basic types
- (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
- will be skipped instead of raising a ``TypeError``.
-
- If ``ensure_ascii`` is ``False``, then the some chunks written to ``fp``
- may be ``unicode`` instances, subject to normal Python ``str`` to
- ``unicode`` coercion rules. Unless ``fp.write()`` explicitly
- understands ``unicode`` (as in ``codecs.getwriter()``) this is likely
- to cause an error.
-
- If ``check_circular`` is ``False``, then the circular reference check
- for container types will be skipped and a circular reference will
- result in an ``OverflowError`` (or worse).
-
- If ``allow_nan`` is ``False``, then it will be a ``ValueError`` to
- serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``)
- in strict compliance of the JSON specification, instead of using the
- JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
-
- If ``indent`` is a non-negative integer, then JSON array elements and object
- members will be pretty-printed with that indent level. An indent level
- of 0 will only insert newlines. ``None`` is the most compact representation.
-
- If ``separators`` is an ``(item_separator, dict_separator)`` tuple
- then it will be used instead of the default ``(', ', ': ')`` separators.
- ``(',', ':')`` is the most compact JSON representation.
-
- ``encoding`` is the character encoding for str instances, default is UTF-8.
-
- ``default(obj)`` is a function that should return a serializable version
- of obj or raise TypeError. The default simply raises TypeError.
-
- To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
- ``.default()`` method to serialize additional types), specify it with
- the ``cls`` kwarg.
-
- """
- # cached encoder
- if (skipkeys is False and ensure_ascii is True and
- check_circular is True and allow_nan is True and
- cls is None and indent is None and separators is None and
- encoding == 'utf-8' and default is None and not kw):
- iterable = _default_encoder.iterencode(obj)
- else:
- if cls is None:
- cls = JSONEncoder
- iterable = cls(skipkeys=skipkeys, ensure_ascii=ensure_ascii,
- check_circular=check_circular, allow_nan=allow_nan, indent=indent,
- separators=separators, encoding=encoding,
- default=default, **kw).iterencode(obj)
- # could accelerate with writelines in some versions of Python, at
- # a debuggability cost
- for chunk in iterable:
- fp.write(chunk)
-
-
-def dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,
- allow_nan=True, cls=None, indent=None, separators=None,
- encoding='utf-8', default=None, **kw):
- """Serialize ``obj`` to a JSON formatted ``str``.
-
- If ``skipkeys`` is ``True`` then ``dict`` keys that are not basic types
- (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
- will be skipped instead of raising a ``TypeError``.
-
- If ``ensure_ascii`` is ``False``, then the return value will be a
- ``unicode`` instance subject to normal Python ``str`` to ``unicode``
- coercion rules instead of being escaped to an ASCII ``str``.
-
- If ``check_circular`` is ``False``, then the circular reference check
- for container types will be skipped and a circular reference will
- result in an ``OverflowError`` (or worse).
-
- If ``allow_nan`` is ``False``, then it will be a ``ValueError`` to
- serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``) in
- strict compliance of the JSON specification, instead of using the
- JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
-
- If ``indent`` is a non-negative integer, then JSON array elements and
- object members will be pretty-printed with that indent level. An indent
- level of 0 will only insert newlines. ``None`` is the most compact
- representation.
-
- If ``separators`` is an ``(item_separator, dict_separator)`` tuple
- then it will be used instead of the default ``(', ', ': ')`` separators.
- ``(',', ':')`` is the most compact JSON representation.
-
- ``encoding`` is the character encoding for str instances, default is UTF-8.
-
- ``default(obj)`` is a function that should return a serializable version
- of obj or raise TypeError. The default simply raises TypeError.
-
- To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
- ``.default()`` method to serialize additional types), specify it with
- the ``cls`` kwarg.
-
- """
- # cached encoder
- if (skipkeys is False and ensure_ascii is True and
- check_circular is True and allow_nan is True and
- cls is None and indent is None and separators is None and
- encoding == 'utf-8' and default is None and not kw):
- return _default_encoder.encode(obj)
- if cls is None:
- cls = JSONEncoder
- return cls(
- skipkeys=skipkeys, ensure_ascii=ensure_ascii,
- check_circular=check_circular, allow_nan=allow_nan, indent=indent,
- separators=separators, encoding=encoding, default=default,
- **kw).encode(obj)
-
-
-_default_decoder = JSONDecoder(encoding=None, object_hook=None)
-
-
-def load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,
- parse_int=None, parse_constant=None, **kw):
- """Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
- a JSON document) to a Python object.
-
- If the contents of ``fp`` is encoded with an ASCII based encoding other
- than utf-8 (e.g. latin-1), then an appropriate ``encoding`` name must
- be specified. Encodings that are not ASCII based (such as UCS-2) are
- not allowed, and should be wrapped with
- ``codecs.getreader(fp)(encoding)``, or simply decoded to a ``unicode``
- object and passed to ``loads()``
-
- ``object_hook`` is an optional function that will be called with the
- result of any object literal decode (a ``dict``). The return value of
- ``object_hook`` will be used instead of the ``dict``. This feature
- can be used to implement custom decoders (e.g. JSON-RPC class hinting).
-
- To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
- kwarg.
-
- """
- return loads(fp.read(),
- encoding=encoding, cls=cls, object_hook=object_hook,
- parse_float=parse_float, parse_int=parse_int,
- parse_constant=parse_constant, **kw)
-
-
-def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
- parse_int=None, parse_constant=None, **kw):
- """Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a JSON
- document) to a Python object.
-
- If ``s`` is a ``str`` instance and is encoded with an ASCII based encoding
- other than utf-8 (e.g. latin-1) then an appropriate ``encoding`` name
- must be specified. Encodings that are not ASCII based (such as UCS-2)
- are not allowed and should be decoded to ``unicode`` first.
-
- ``object_hook`` is an optional function that will be called with the
- result of any object literal decode (a ``dict``). The return value of
- ``object_hook`` will be used instead of the ``dict``. This feature
- can be used to implement custom decoders (e.g. JSON-RPC class hinting).
-
- ``parse_float``, if specified, will be called with the string
- of every JSON float to be decoded. By default this is equivalent to
- float(num_str). This can be used to use another datatype or parser
- for JSON floats (e.g. decimal.Decimal).
-
- ``parse_int``, if specified, will be called with the string
- of every JSON int to be decoded. By default this is equivalent to
- int(num_str). This can be used to use another datatype or parser
- for JSON integers (e.g. float).
-
- ``parse_constant``, if specified, will be called with one of the
- following strings: -Infinity, Infinity, NaN, null, true, false.
- This can be used to raise an exception if invalid JSON numbers
- are encountered.
-
- To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
- kwarg.
-
- """
- if (cls is None and encoding is None and object_hook is None and
- parse_int is None and parse_float is None and
- parse_constant is None and not kw):
- return _default_decoder.decode(s)
- if cls is None:
- cls = JSONDecoder
- if object_hook is not None:
- kw['object_hook'] = object_hook
- if parse_float is not None:
- kw['parse_float'] = parse_float
- if parse_int is not None:
- kw['parse_int'] = parse_int
- if parse_constant is not None:
- kw['parse_constant'] = parse_constant
- return cls(encoding=encoding, **kw).decode(s)
View
2,265 simplejson/_speedups.c
@@ -1,2265 +0,0 @@
-#include "Python.h"
-#include "structmember.h"
-#if PY_VERSION_HEX < 0x02060000 && !defined(Py_TYPE)
-#define Py_TYPE(ob) (((PyObject*)(ob))->ob_type)
-#endif
-#if PY_VERSION_HEX < 0x02050000 && !defined(PY_SSIZE_T_MIN)
-typedef int Py_ssize_t;
-#define PY_SSIZE_T_MAX INT_MAX
-#define PY_SSIZE_T_MIN INT_MIN
-#define PyInt_FromSsize_t PyInt_FromLong
-#define PyInt_AsSsize_t PyInt_AsLong
-#endif
-#ifndef Py_IS_FINITE
-#define Py_IS_FINITE(X) (!Py_IS_INFINITY(X) && !Py_IS_NAN(X))
-#endif
-
-#ifdef __GNUC__
-#define UNUSED __attribute__((__unused__))
-#else
-#define UNUSED
-#endif
-
-#define DEFAULT_ENCODING "utf-8"
-
-#define PyScanner_Check(op) PyObject_TypeCheck(op, &PyScannerType)
-#define PyScanner_CheckExact(op) (Py_TYPE(op) == &PyScannerType)
-#define PyEncoder_Check(op) PyObject_TypeCheck(op, &PyEncoderType)
-#define PyEncoder_CheckExact(op) (Py_TYPE(op) == &PyEncoderType)
-
-static PyTypeObject PyScannerType;
-static PyTypeObject PyEncoderType;
-
-typedef struct _PyScannerObject {
- PyObject_HEAD
- PyObject *encoding;
- PyObject *strict;
- PyObject *object_hook;
- PyObject *parse_float;
- PyObject *parse_int;
- PyObject *parse_constant;
-} PyScannerObject;
-
-static PyMemberDef scanner_members[] = {
- {"encoding", T_OBJECT, offsetof(PyScannerObject, encoding), READONLY, "encoding"},
- {"strict", T_OBJECT, offsetof(PyScannerObject, strict), READONLY, "strict"},
- {"object_hook", T_OBJECT, offsetof(PyScannerObject, object_hook), READONLY, "object_hook"},
- {"parse_float", T_OBJECT, offsetof(PyScannerObject, parse_float), READONLY, "parse_float"},
- {"parse_int", T_OBJECT, offsetof(PyScannerObject, parse_int), READONLY, "parse_int"},
- {"parse_constant", T_OBJECT, offsetof(PyScannerObject, parse_constant), READONLY, "parse_constant"},
- {NULL}
-};
-
-typedef struct _PyEncoderObject {
- PyObject_HEAD
- PyObject *markers;
- PyObject *defaultfn;
- PyObject *encoder;
- PyObject *indent;
- PyObject *key_separator;
- PyObject *item_separator;
- PyObject *sort_keys;
- PyObject *skipkeys;
- int fast_encode;
- int allow_nan;
-} PyEncoderObject;
-
-static PyMemberDef encoder_members[] = {
- {"markers", T_OBJECT, offsetof(PyEncoderObject, markers), READONLY, "markers"},
- {"default", T_OBJECT, offsetof(PyEncoderObject, defaultfn), READONLY, "default"},
- {"encoder", T_OBJECT, offsetof(PyEncoderObject, encoder), READONLY, "encoder"},
- {"indent", T_OBJECT, offsetof(PyEncoderObject, indent), READONLY, "indent"},
- {"key_separator", T_OBJECT, offsetof(PyEncoderObject, key_separator), READONLY, "key_separator"},
- {"item_separator", T_OBJECT, offsetof(PyEncoderObject, item_separator), READONLY, "item_separator"},
- {"sort_keys", T_OBJECT, offsetof(PyEncoderObject, sort_keys), READONLY, "sort_keys"},
- {"skipkeys", T_OBJECT, offsetof(PyEncoderObject, skipkeys), READONLY, "skipkeys"},
- {NULL}
-};
-
-static Py_ssize_t
-ascii_escape_char(Py_UNICODE c, char *output, Py_ssize_t chars);
-static PyObject *
-ascii_escape_unicode(PyObject *pystr);
-static PyObject *
-ascii_escape_str(PyObject *pystr);
-static PyObject *
-py_encode_basestring_ascii(PyObject* self UNUSED, PyObject *pystr);
-void init_speedups(void);
-static PyObject *
-scan_once_str(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr);
-static PyObject *
-scan_once_unicode(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr);
-static PyObject *
-_build_rval_index_tuple(PyObject *rval, Py_ssize_t idx);
-static int
-scanner_init(PyObject *self, PyObject *args, PyObject *kwds);
-static void
-scanner_dealloc(PyObject *self);
-static int
-encoder_init(PyObject *self, PyObject *args, PyObject *kwds);
-static void
-encoder_dealloc(PyObject *self);
-static int
-encoder_listencode_list(PyEncoderObject *s, PyObject *rval, PyObject *seq, Py_ssize_t indent_level);
-static int
-encoder_listencode_obj(PyEncoderObject *s, PyObject *rval, PyObject *obj, Py_ssize_t indent_level);
-static int
-encoder_listencode_dict(PyEncoderObject *s, PyObject *rval, PyObject *dct, Py_ssize_t indent_level);
-static PyObject *
-_encoded_const(PyObject *const);
-static void
-raise_errmsg(char *msg, PyObject *s, Py_ssize_t end);
-static PyObject *
-encoder_encode_string(PyEncoderObject *s, PyObject *obj);
-static int
-_convertPyInt_AsSsize_t(PyObject *o, Py_ssize_t *size_ptr);
-static PyObject *
-_convertPyInt_FromSsize_t(Py_ssize_t *size_ptr);
-static PyObject *
-encoder_encode_float(PyEncoderObject *s, PyObject *obj);
-
-#define S_CHAR(c) (c >= ' ' && c <= '~' && c != '\\' && c != '"')
-#define IS_WHITESPACE(c) (((c) == ' ') || ((c) == '\t') || ((c) == '\n') || ((c) == '\r'))
-
-#define MIN_EXPANSION 6
-#ifdef Py_UNICODE_WIDE
-#define MAX_EXPANSION (2 * MIN_EXPANSION)
-#else
-#define MAX_EXPANSION MIN_EXPANSION
-#endif
-
-static int
-_convertPyInt_AsSsize_t(PyObject *o, Py_ssize_t *size_ptr)
-{
- /* PyObject to Py_ssize_t converter */
- *size_ptr = PyInt_AsSsize_t(o);
- if (*size_ptr == -1 && PyErr_Occurred());
- return 1;
- return 0;
-}
-
-static PyObject *
-_convertPyInt_FromSsize_t(Py_ssize_t *size_ptr)
-{
- /* Py_ssize_t to PyObject converter */
- return PyInt_FromSsize_t(*size_ptr);
-}
-
-static Py_ssize_t
-ascii_escape_char(Py_UNICODE c, char *output, Py_ssize_t chars)
-{
- /* Escape unicode code point c to ASCII escape sequences
- in char *output. output must have at least 12 bytes unused to
- accommodate an escaped surrogate pair "\uXXXX\uXXXX" */
- output[chars++] = '\\';
- switch (c) {
- case '\\': output[chars++] = (char)c; break;
- case '"': output[chars++] = (char)c; break;
- case '\b': output[chars++] = 'b'; break;
- case '\f': output[chars++] = 'f'; break;
- case '\n': output[chars++] = 'n'; break;
- case '\r': output[chars++] = 'r'; break;
- case '\t': output[chars++] = 't'; break;
- default:
-#ifdef Py_UNICODE_WIDE
- if (c >= 0x10000) {
- /* UTF-16 surrogate pair */
- Py_UNICODE v = c - 0x10000;
- c = 0xd800 | ((v >> 10) & 0x3ff);
- output[chars++] = 'u';
- output[chars++] = "0123456789abcdef"[(c >> 12) & 0xf];
- output[chars++] = "0123456789abcdef"[(c >> 8) & 0xf];
- output[chars++] = "0123456789abcdef"[(c >> 4) & 0xf];
- output[chars++] = "0123456789abcdef"[(c ) & 0xf];
- c = 0xdc00 | (v & 0x3ff);
- output[chars++] = '\\';
- }
-#endif
- output[chars++] = 'u';
- output[chars++] = "0123456789abcdef"[(c >> 12) & 0xf];
- output[chars++] = "0123456789abcdef"[(c >> 8) & 0xf];
- output[chars++] = "0123456789abcdef"[(c >> 4) & 0xf];
- output[chars++] = "0123456789abcdef"[(c ) & 0xf];
- }
- return chars;
-}
-
-static PyObject *
-ascii_escape_unicode(PyObject *pystr)
-{
- /* Take a PyUnicode pystr and return a new ASCII-only escaped PyString */
- Py_ssize_t i;
- Py_ssize_t input_chars;
- Py_ssize_t output_size;
- Py_ssize_t max_output_size;
- Py_ssize_t chars;
- PyObject *rval;
- char *output;
- Py_UNICODE *input_unicode;
-
- input_chars = PyUnicode_GET_SIZE(pystr);
- input_unicode = PyUnicode_AS_UNICODE(pystr);
-
- /* One char input can be up to 6 chars output, estimate 4 of these */
- output_size = 2 + (MIN_EXPANSION * 4) + input_chars;
- max_output_size = 2 + (input_chars * MAX_EXPANSION);
- rval = PyString_FromStringAndSize(NULL, output_size);
- if (rval == NULL) {
- return NULL;
- }
- output = PyString_AS_STRING(rval);
- chars = 0;
- output[chars++] = '"';
- for (i = 0; i < input_chars; i++) {
- Py_UNICODE c = input_unicode[i];
- if (S_CHAR(c)) {
- output[chars++] = (char)c;
- }
- else {
- chars = ascii_escape_char(c, output, chars);
- }
- if (output_size - chars < (1 + MAX_EXPANSION)) {
- /* There's more than four, so let's resize by a lot */
- Py_ssize_t new_output_size = output_size * 2;
- /* This is an upper bound */
- if (new_output_size > max_output_size) {
- new_output_size = max_output_size;
- }
- /* Make sure that the output size changed before resizing */
- if (new_output_size != output_size) {
- output_size = new_output_size;
- if (_PyString_Resize(&rval, output_size) == -1) {
- return NULL;
- }
- output = PyString_AS_STRING(rval);
- }
- }
- }
- output[chars++] = '"';
- if (_PyString_Resize(&rval, chars) == -1) {
- return NULL;
- }
- return rval;
-}
-
-static PyObject *
-ascii_escape_str(PyObject *pystr)
-{
- /* Take a PyString pystr and return a new ASCII-only escaped PyString */
- Py_ssize_t i;
- Py_ssize_t input_chars;
- Py_ssize_t output_size;
- Py_ssize_t chars;
- PyObject *rval;
- char *output;
- char *input_str;
-
- input_chars = PyString_GET_SIZE(pystr);
- input_str = PyString_AS_STRING(pystr);
-
- /* Fast path for a string that's already ASCII */
- for (i = 0; i < input_chars; i++) {
- Py_UNICODE c = (Py_UNICODE)(unsigned char)input_str[i];
- if (!S_CHAR(c)) {
- /* If we have to escape something, scan the string for unicode */
- Py_ssize_t j;
- for (j = i; j < input_chars; j++) {
- c = (Py_UNICODE)(unsigned char)input_str[j];
- if (c > 0x7f) {
- /* We hit a non-ASCII character, bail to unicode mode */
- PyObject *uni;
- uni = PyUnicode_DecodeUTF8(input_str, input_chars, "strict");
- if (uni == NULL) {
- return NULL;
- }
- rval = ascii_escape_unicode(uni);
- Py_DECREF(uni);
- return rval;
- }
- }
- break;
- }
- }
-
- if (i == input_chars) {
- /* Input is already ASCII */
- output_size = 2 + input_chars;
- }
- else {
- /* One char input can be up to 6 chars output, estimate 4 of these */
- output_size = 2 + (MIN_EXPANSION * 4) + input_chars;
- }
- rval = PyString_FromStringAndSize(NULL, output_size);
- if (rval == NULL) {
- return NULL;
- }
- output = PyString_AS_STRING(rval);
- output[0] = '"';
-
- /* We know that everything up to i is ASCII already */
- chars = i + 1;
- memcpy(&output[1], input_str, i);
-
- for (; i < input_chars; i++) {
- Py_UNICODE c = (Py_UNICODE)(unsigned char)input_str[i];
- if (S_CHAR(c)) {
- output[chars++] = (char)c;
- }
- else {
- chars = ascii_escape_char(c, output, chars);
- }
- /* An ASCII char can't possibly expand to a surrogate! */
- if (output_size - chars < (1 + MIN_EXPANSION)) {
- /* There's more than four, so let's resize by a lot */
- output_size *= 2;
- if (output_size > 2 + (input_chars * MIN_EXPANSION)) {
- output_size = 2 + (input_chars * MIN_EXPANSION);
- }
- if (_PyString_Resize(&rval, output_size) == -1) {
- return NULL;
- }
- output = PyString_AS_STRING(rval);
- }
- }
- output[chars++] = '"';
- if (_PyString_Resize(&rval, chars) == -1) {
- return NULL;
- }
- return rval;
-}
-
-static void
-raise_errmsg(char *msg, PyObject *s, Py_ssize_t end)
-{
- /* Use the Python function simplejson.decoder.errmsg to raise a nice
- looking ValueError exception */
- static PyObject *errmsg_fn = NULL;
- PyObject *pymsg;
- if (errmsg_fn == NULL) {
- PyObject *decoder = PyImport_ImportModule("simplejson.decoder");
- if (decoder == NULL)
- return;
- errmsg_fn = PyObject_GetAttrString(decoder, "errmsg");
- Py_DECREF(decoder);
- if (errmsg_fn == NULL)
- return;
- }
- pymsg = PyObject_CallFunction(errmsg_fn, "(zOO&)", msg, s, _convertPyInt_FromSsize_t, &end);
- if (pymsg) {
- PyErr_SetObject(PyExc_ValueError, pymsg);
- Py_DECREF(pymsg);
- }
-}
-
-static PyObject *
-join_list_unicode(PyObject *lst)
-{
- /* return u''.join(lst) */
- static PyObject *joinfn = NULL;
- if (joinfn == NULL) {
- PyObject *ustr = PyUnicode_FromUnicode(NULL, 0);
- if (ustr == NULL)
- return NULL;
-
- joinfn = PyObject_GetAttrString(ustr, "join");
- Py_DECREF(ustr);
- if (joinfn == NULL)
- return NULL;
- }
- return PyObject_CallFunctionObjArgs(joinfn, lst, NULL);
-}
-
-static PyObject *
-join_list_string(PyObject *lst)
-{
- /* return ''.join(lst) */
- static PyObject *joinfn = NULL;
- if (joinfn == NULL) {
- PyObject *ustr = PyString_FromStringAndSize(NULL, 0);
- if (ustr == NULL)
- return NULL;
-
- joinfn = PyObject_GetAttrString(ustr, "join");
- Py_DECREF(ustr);
- if (joinfn == NULL)
- return NULL;
- }
- return PyObject_CallFunctionObjArgs(joinfn, lst, NULL);
-}
-
-static PyObject *
-_build_rval_index_tuple(PyObject *rval, Py_ssize_t idx) {
- /* return (rval, idx) tuple, stealing reference to rval */
- PyObject *tpl;
- PyObject *pyidx;
- /*
- steal a reference to rval, returns (rval, idx)
- */
- if (rval == NULL) {
- return NULL;
- }
- pyidx = PyInt_FromSsize_t(idx);
- if (pyidx == NULL) {
- Py_DECREF(rval);
- return NULL;
- }
- tpl = PyTuple_New(2);
- if (tpl == NULL) {
- Py_DECREF(pyidx);
- Py_DECREF(rval);
- return NULL;
- }
- PyTuple_SET_ITEM(tpl, 0, rval);
- PyTuple_SET_ITEM(tpl, 1, pyidx);
- return tpl;
-}
-
-static PyObject *
-scanstring_str(PyObject *pystr, Py_ssize_t end, char *encoding, int strict, Py_ssize_t *next_end_ptr)
-{
- /* Read the JSON string from PyString pystr.
- end is the index of the first character after the quote.
- encoding is the encoding of pystr (must be an ASCII superset)
- if strict is zero then literal control characters are allowed
- *next_end_ptr is a return-by-reference index of the character
- after the end quote
-
- Return value is a new PyString (if ASCII-only) or PyUnicode
- */
- PyObject *rval;
- Py_ssize_t len = PyString_GET_SIZE(pystr);
- Py_ssize_t begin = end - 1;
- Py_ssize_t next = begin;
- int has_unicode = 0;
- char *buf = PyString_AS_STRING(pystr);
- PyObject *chunks = PyList_New(0);
- if (chunks == NULL) {
- goto bail;
- }
- if (end < 0 || len <= end) {
- PyErr_SetString(PyExc_ValueError, "end is out of bounds");
- goto bail;
- }
- while (1) {
- /* Find the end of the string or the next escape */
- Py_UNICODE c = 0;
- PyObject *chunk = NULL;
- for (next = end; next < len; next++) {
- c = (unsigned char)buf[next];
- if (c == '"' || c == '\\') {
- break;
- }
- else if (strict && c <= 0x1f) {
- raise_errmsg("Invalid control character at", pystr, next);
- goto bail;
- }
- else if (c > 0x7f) {
- has_unicode = 1;
- }
- }
- if (!(c == '"' || c == '\\')) {
- raise_errmsg("Unterminated string starting at", pystr, begin);
- goto bail;
- }
- /* Pick up this chunk if it's not zero length */
- if (next != end) {
- PyObject *strchunk = PyString_FromStringAndSize(&buf[end], next - end);
- if (strchunk == NULL) {
- goto bail;
- }
- if (has_unicode) {
- chunk = PyUnicode_FromEncodedObject(strchunk, encoding, NULL);
- Py_DECREF(strchunk);
- if (chunk == NULL) {
- goto bail;
- }
- }
- else {
- chunk = strchunk;
- }
- if (PyList_Append(chunks, chunk)) {
- Py_DECREF(chunk);
- goto bail;
- }
- Py_DECREF(chunk);
- }
- next++;
- if (c == '"') {
- end = next;
- break;
- }
- if (next == len) {
- raise_errmsg("Unterminated string starting at", pystr, begin);
- goto bail;
- }
- c = buf[next];
- if (c != 'u') {
- /* Non-unicode backslash escapes */
- end = next + 1;
- switch (c) {
- case '"': break;
- case '\\': break;
- case '/': break;
- case 'b': c = '\b'; break;
- case 'f': c = '\f'; break;
- case 'n': c = '\n'; break;
- case 'r': c = '\r'; break;
- case 't': c = '\t'; break;
- default: c = 0;
- }
- if (c == 0) {
- raise_errmsg("Invalid \\escape", pystr, end - 2);
- goto bail;
- }
- }
- else {
- c = 0;
- next++;
- end = next + 4;
- if (end >= len) {
- raise_errmsg("Invalid \\uXXXX escape", pystr, next - 1);
- goto bail;
- }
- /* Decode 4 hex digits */
- for (; next < end; next++) {
- Py_UNICODE digit = buf[next];
- c <<= 4;
- switch (digit) {
- case '0': case '1': case '2': case '3': case '4':
- case '5': case '6': case '7': case '8': case '9':
- c |= (digit - '0'); break;
- case 'a': case 'b': case 'c': case 'd': case 'e':
- case 'f':
- c |= (digit - 'a' + 10); break;
- case 'A': case 'B': case 'C': case 'D': case 'E':
- case 'F':
- c |= (digit - 'A' + 10); break;
- default:
- raise_errmsg("Invalid \\uXXXX escape", pystr, end - 5);
- goto bail;
- }
- }
-#ifdef Py_UNICODE_WIDE
- /* Surrogate pair */
- if ((c & 0xfc00) == 0xd800) {
- Py_UNICODE c2 = 0;
- if (end + 6 >= len) {
- raise_errmsg("Unpaired high surrogate", pystr, end - 5);
- goto bail;
- }
- if (buf[next++] != '\\' || buf[next++] != 'u') {
- raise_errmsg("Unpaired high surrogate", pystr, end - 5);
- goto bail;
- }
- end += 6;
- /* Decode 4 hex digits */
- for (; next < end; next++) {
- c2 <<= 4;
- Py_UNICODE digit = buf[next];
- switch (digit) {
- case '0': case '1': case '2': case '3': case '4':
- case '5': case '6': case '7': case '8': case '9':
- c2 |= (digit - '0'); break;
- case 'a': case 'b': case 'c': case 'd': case 'e':
- case 'f':
- c2 |= (digit - 'a' + 10); break;
- case 'A': case 'B': case 'C': case 'D': case 'E':
- case 'F':
- c2 |= (digit - 'A' + 10); break;
- default:
- raise_errmsg("Invalid \\uXXXX escape", pystr, end - 5);
- goto bail;
- }
- }
- if ((c2 & 0xfc00) != 0xdc00) {
- raise_errmsg("Unpaired high surrogate", pystr, end - 5);
- goto bail;
- }
- c = 0x10000 + (((c - 0xd800) << 10) | (c2 - 0xdc00));
- }
- else if ((c & 0xfc00) == 0xdc00) {
- raise_errmsg("Unpaired low surrogate", pystr, end - 5);
- goto bail;
- }
-#endif
- }
- if (c > 0x7f) {
- has_unicode = 1;
- }
- if (has_unicode) {
- chunk = PyUnicode_FromUnicode(&c, 1);
- if (chunk == NULL) {
- goto bail;
- }
- }
- else {
- char c_char = Py_CHARMASK(c);
- chunk = PyString_FromStringAndSize(&c_char, 1);
- if (chunk == NULL) {
- goto bail;
- }
- }
- if (PyList_Append(chunks, chunk)) {
- Py_DECREF(chunk);
- goto bail;
- }
- Py_DECREF(chunk);
- }
-
- rval = join_list_string(chunks);
- if (rval == NULL) {
- goto bail;
- }
- Py_CLEAR(chunks);
- *next_end_ptr = end;
- return rval;
-bail:
- *next_end_ptr = -1;
- Py_XDECREF(chunks);
- return NULL;
-}
-
-
-static PyObject *
-scanstring_unicode(PyObject *pystr, Py_ssize_t end, int strict, Py_ssize_t *next_end_ptr)
-{
- /* Read the JSON string from PyUnicode pystr.
- end is the index of the first character after the quote.
- encoding is the encoding of pystr (must be an ASCII superset)
- if strict is zero then literal control characters are allowed
- *next_end_ptr is a return-by-reference index of the character
- after the end quote
-
- Return value is a new PyUnicode
- */
- PyObject *rval;
- Py_ssize_t len = PyUnicode_GET_SIZE(pystr);
- Py_ssize_t begin = end - 1;
- Py_ssize_t next = begin;
- const Py_UNICODE *buf = PyUnicode_AS_UNICODE(pystr);
- PyObject *chunks = PyList_New(0);
- if (chunks == NULL) {
- goto bail;
- }
- if (end < 0 || len <= end) {
- PyErr_SetString(PyExc_ValueError, "end is out of bounds");
- goto bail;
- }
- while (1) {
- /* Find the end of the string or the next escape */
- Py_UNICODE c = 0;
- PyObject *chunk = NULL;
- for (next = end; next < len; next++) {
- c = buf[next];
- if (c == '"' || c == '\\') {
- break;
- }
- else if (strict && c <= 0x1f) {
- raise_errmsg("Invalid control character at", pystr, next);
- goto bail;
- }
- }
- if (!(c == '"' || c == '\\')) {
- raise_errmsg("Unterminated string starting at", pystr, begin);
- goto bail;
- }
- /* Pick up this chunk if it's not zero length */
- if (next != end) {
- chunk = PyUnicode_FromUnicode(&buf[end], next - end);
- if (chunk == NULL) {
- goto bail;
- }
- if (PyList_Append(chunks, chunk)) {
- Py_DECREF(chunk);
- goto bail;
- }
- Py_DECREF(chunk);
- }
- next++;
- if (c == '"') {
- end = next;
- break;
- }
- if (next == len) {
- raise_errmsg("Unterminated string starting at", pystr, begin);
- goto bail;
- }
- c = buf[next];
- if (c != 'u') {
- /* Non-unicode backslash escapes */
- end = next + 1;
- switch (c) {
- case '"': break;
- case '\\': break;
- case '/': break;
- case 'b': c = '\b'; break;
- case 'f': c = '\f'; break;
- case 'n': c = '\n'; break;
- case 'r': c = '\r'; break;
- case 't': c = '\t'; break;
- default: c = 0;
- }
- if (c == 0) {
- raise_errmsg("Invalid \\escape", pystr, end - 2);
- goto bail;
- }
- }
- else {
- c = 0;
- next++;
- end = next + 4;
- if (end >= len) {
- raise_errmsg("Invalid \\uXXXX escape", pystr, next - 1);
- goto bail;
- }
- /* Decode 4 hex digits */
- for (; next < end; next++) {
- Py_UNICODE digit = buf[next];
- c <<= 4;
- switch (digit) {
- case '0': case '1': case '2': case '3': case '4':
- case '5': case '6': case '7': case '8': case '9':
- c |= (digit - '0'); break;
- case 'a': case 'b': case 'c': case 'd': case 'e':
- case 'f':
- c |= (digit - 'a' + 10); break;
- case 'A': case 'B': case 'C': case 'D': case 'E':
- case 'F':
- c |= (digit - 'A' + 10); break;
- default:
- raise_errmsg("Invalid \\uXXXX escape", pystr, end - 5);
- goto bail;
- }
- }
-#ifdef Py_UNICODE_WIDE
- /* Surrogate pair */
- if ((c & 0xfc00) == 0xd800) {
- Py_UNICODE c2 = 0;
- if (end + 6 >= len) {
- raise_errmsg("Unpaired high surrogate", pystr, end - 5);
- goto bail;
- }
- if (buf[next++] != '\\' || buf[next++] != 'u') {
- raise_errmsg("Unpaired high surrogate", pystr, end - 5);
- goto bail;
- }
- end += 6;
- /* Decode 4 hex digits */
- for (; next < end; next++) {
- c2 <<= 4;
- Py_UNICODE digit = buf[next];
- switch (digit) {
- case '0': case '1': case '2': case '3': case '4':
- case '5': case '6': case '7': case '8': case '9':
- c2 |= (digit - '0'); break;
- case 'a': case 'b': case 'c': case 'd': case 'e':
- case 'f':
- c2 |= (digit - 'a' + 10); break;
- case 'A': case 'B': case 'C': case 'D': case 'E':
- case 'F':
- c2 |= (digit - 'A' + 10); break;
- default:
- raise_errmsg("Invalid \\uXXXX escape", pystr, end - 5);
- goto bail;
- }
- }
- if ((c2 & 0xfc00) != 0xdc00) {
- raise_errmsg("Unpaired high surrogate", pystr, end - 5);
- goto bail;
- }
- c = 0x10000 + (((c - 0xd800) << 10) | (c2 - 0xdc00));
- }
- else if ((c & 0xfc00) == 0xdc00) {
- raise_errmsg("Unpaired low surrogate", pystr, end - 5);
- goto bail;
- }
-#endif
- }
- chunk = PyUnicode_FromUnicode(&c, 1);
- if (chunk == NULL) {
- goto bail;
- }
- if (PyList_Append(chunks, chunk)) {
- Py_DECREF(chunk);
- goto bail;
- }
- Py_DECREF(chunk);
- }
-
- rval = join_list_unicode(chunks);
- if (rval == NULL) {
- goto bail;
- }
- Py_DECREF(chunks);
- *next_end_ptr = end;
- return rval;
-bail:
- *next_end_ptr = -1;
- Py_XDECREF(chunks);
- return NULL;
-}
-
-PyDoc_STRVAR(pydoc_scanstring,
- "scanstring(basestring, end, encoding, strict=True) -> (str, end)\n"
- "\n"
- "Scan the string s for a JSON string. End is the index of the\n"
- "character in s after the quote that started the JSON string.\n"
- "Unescapes all valid JSON string escape sequences and raises ValueError\n"
- "on attempt to decode an invalid string. If strict is False then literal\n"
- "control characters are allowed in the string.\n"
- "\n"
- "Returns a tuple of the decoded string and the index of the character in s\n"
- "after the end quote."
-);
-
-static PyObject *
-py_scanstring(PyObject* self UNUSED, PyObject *args)
-{
- PyObject *pystr;
- PyObject *rval;
- Py_ssize_t end;
- Py_ssize_t next_end = -1;
- char *encoding = NULL;
- int strict = 1;
- if (!PyArg_ParseTuple(args, "OO&|zi:scanstring", &pystr, _convertPyInt_AsSsize_t, &end, &encoding, &strict)) {
- return NULL;
- }
- if (encoding == NULL) {
- encoding = DEFAULT_ENCODING;
- }
- if (PyString_Check(pystr)) {
- rval = scanstring_str(pystr, end, encoding, strict, &next_end);
- }
- else if (PyUnicode_Check(pystr)) {
- rval = scanstring_unicode(pystr, end, strict, &next_end);
- }
- else {
- PyErr_Format(PyExc_TypeError,
- "first argument must be a string, not %.80s",
- Py_TYPE(pystr)->tp_name);
- return NULL;
- }
- return _build_rval_index_tuple(rval, next_end);
-}
-
-PyDoc_STRVAR(pydoc_encode_basestring_ascii,
- "encode_basestring_ascii(basestring) -> str\n"
- "\n"
- "Return an ASCII-only JSON representation of a Python string"
-);
-
-static PyObject *
-py_encode_basestring_ascii(PyObject* self UNUSED, PyObject *pystr)
-{
- /* Return an ASCII-only JSON representation of a Python string */
- /* METH_O */
- if (PyString_Check(pystr)) {
- return ascii_escape_str(pystr);
- }
- else if (PyUnicode_Check(pystr)) {
- return ascii_escape_unicode(pystr);
- }
- else {
- PyErr_Format(PyExc_TypeError,
- "first argument must be a string, not %.80s",
- Py_TYPE(pystr)->tp_name);
- return NULL;
- }
-}
-
-static void
-scanner_dealloc(PyObject *self)
-{
- /* Deallocate scanner object */
- PyScannerObject *s;
- assert(PyScanner_Check(self));
- s = (PyScannerObject *)self;
- Py_CLEAR(s->encoding);
- Py_CLEAR(s->strict);
- Py_CLEAR(s->object_hook);
- Py_CLEAR(s->parse_float);
- Py_CLEAR(s->parse_int);
- Py_CLEAR(s->parse_constant);
- self->ob_type->tp_free(self);
-}
-
-static PyObject *
-_parse_object_str(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr) {
- /* Read a JSON object from PyString pystr.
- idx is the index of the first character after the opening curly brace.
- *next_idx_ptr is a return-by-reference index to the first character after
- the closing curly brace.
-
- Returns a new PyObject (usually a dict, but object_hook can change that)
- */
- char *str = PyString_AS_STRING(pystr);
- Py_ssize_t end_idx = PyString_GET_SIZE(pystr) - 1;
- PyObject *rval = PyDict_New();
- PyObject *key = NULL;
- PyObject *val = NULL;
- char *encoding = PyString_AS_STRING(s->encoding);
- int strict = PyObject_IsTrue(s->strict);
- Py_ssize_t next_idx;
- if (rval == NULL)
- return NULL;
-
- /* skip whitespace after { */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* only loop if the object is non-empty */
- if (idx <= end_idx && str[idx] != '}') {
- while (idx <= end_idx) {
- /* read key */
- if (str[idx] != '"') {
- raise_errmsg("Expecting property name", pystr, idx);
- goto bail;
- }
- key = scanstring_str(pystr, idx + 1, encoding, strict, &next_idx);
- if (key == NULL)
- goto bail;
- idx = next_idx;
-
- /* skip whitespace between key and : delimiter, read :, skip whitespace */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
- if (idx > end_idx || str[idx] != ':') {
- raise_errmsg("Expecting : delimiter", pystr, idx);
- goto bail;
- }
- idx++;
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* read any JSON data type */
- val = scan_once_str(s, pystr, idx, &next_idx);
- if (val == NULL)
- goto bail;
-
- if (PyDict_SetItem(rval, key, val) == -1)
- goto bail;
-
- Py_CLEAR(key);
- Py_CLEAR(val);
- idx = next_idx;
-
- /* skip whitespace before } or , */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* bail if the object is closed or we didn't get the , delimiter */
- if (idx > end_idx) break;
- if (str[idx] == '}') {
- break;
- }
- else if (str[idx] != ',') {
- raise_errmsg("Expecting , delimiter", pystr, idx);
- goto bail;
- }
- idx++;
-
- /* skip whitespace after , delimiter */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
- }
- }
- /* verify that idx < end_idx, str[idx] should be '}' */
- if (idx > end_idx || str[idx] != '}') {
- raise_errmsg("Expecting object", pystr, end_idx);
- goto bail;
- }
- /* if object_hook is not None: rval = object_hook(rval) */
- if (s->object_hook != Py_None) {
- val = PyObject_CallFunctionObjArgs(s->object_hook, rval, NULL);
- if (val == NULL)
- goto bail;
- Py_DECREF(rval);
- rval = val;
- val = NULL;
- }
- *next_idx_ptr = idx + 1;
- return rval;
-bail:
- Py_XDECREF(key);
- Py_XDECREF(val);
- Py_DECREF(rval);
- return NULL;
-}
-
-static PyObject *
-_parse_object_unicode(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr) {
- /* Read a JSON object from PyUnicode pystr.
- idx is the index of the first character after the opening curly brace.
- *next_idx_ptr is a return-by-reference index to the first character after
- the closing curly brace.
-
- Returns a new PyObject (usually a dict, but object_hook can change that)
- */
- Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr);
- Py_ssize_t end_idx = PyUnicode_GET_SIZE(pystr) - 1;
- PyObject *val = NULL;
- PyObject *rval = PyDict_New();
- PyObject *key = NULL;
- int strict = PyObject_IsTrue(s->strict);
- Py_ssize_t next_idx;
- if (rval == NULL)
- return NULL;
-
- /* skip whitespace after { */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* only loop if the object is non-empty */
- if (idx <= end_idx && str[idx] != '}') {
- while (idx <= end_idx) {
- /* read key */
- if (str[idx] != '"') {
- raise_errmsg("Expecting property name", pystr, idx);
- goto bail;
- }
- key = scanstring_unicode(pystr, idx + 1, strict, &next_idx);
- if (key == NULL)
- goto bail;
- idx = next_idx;
-
- /* skip whitespace between key and : delimiter, read :, skip whitespace */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
- if (idx > end_idx || str[idx] != ':') {
- raise_errmsg("Expecting : delimiter", pystr, idx);
- goto bail;
- }
- idx++;
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* read any JSON term */
- val = scan_once_unicode(s, pystr, idx, &next_idx);
- if (val == NULL)
- goto bail;
-
- if (PyDict_SetItem(rval, key, val) == -1)
- goto bail;
-
- Py_CLEAR(key);
- Py_CLEAR(val);
- idx = next_idx;
-
- /* skip whitespace before } or , */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* bail if the object is closed or we didn't get the , delimiter */
- if (idx > end_idx) break;
- if (str[idx] == '}') {
- break;
- }
- else if (str[idx] != ',') {
- raise_errmsg("Expecting , delimiter", pystr, idx);
- goto bail;
- }
- idx++;
-
- /* skip whitespace after , delimiter */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
- }
- }
-
- /* verify that idx < end_idx, str[idx] should be '}' */
- if (idx > end_idx || str[idx] != '}') {
- raise_errmsg("Expecting object", pystr, end_idx);
- goto bail;
- }
-
- /* if object_hook is not None: rval = object_hook(rval) */
- if (s->object_hook != Py_None) {
- val = PyObject_CallFunctionObjArgs(s->object_hook, rval, NULL);
- if (val == NULL)
- goto bail;
- Py_DECREF(rval);
- rval = val;
- val = NULL;
- }
- *next_idx_ptr = idx + 1;
- return rval;
-bail:
- Py_XDECREF(key);
- Py_XDECREF(val);
- Py_DECREF(rval);
- return NULL;
-}
-
-static PyObject *
-_parse_array_str(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr) {
- /* Read a JSON array from PyString pystr.
- idx is the index of the first character after the opening brace.
- *next_idx_ptr is a return-by-reference index to the first character after
- the closing brace.
-
- Returns a new PyList
- */
- char *str = PyString_AS_STRING(pystr);
- Py_ssize_t end_idx = PyString_GET_SIZE(pystr) - 1;
- PyObject *val = NULL;
- PyObject *rval = PyList_New(0);
- Py_ssize_t next_idx;
- if (rval == NULL)
- return NULL;
-
- /* skip whitespace after [ */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* only loop if the array is non-empty */
- if (idx <= end_idx && str[idx] != ']') {
- while (idx <= end_idx) {
-
- /* read any JSON term and de-tuplefy the (rval, idx) */
- val = scan_once_str(s, pystr, idx, &next_idx);
- if (val == NULL)
- goto bail;
-
- if (PyList_Append(rval, val) == -1)
- goto bail;
-
- Py_CLEAR(val);
- idx = next_idx;
-
- /* skip whitespace between term and , */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* bail if the array is closed or we didn't get the , delimiter */
- if (idx > end_idx) break;
- if (str[idx] == ']') {
- break;
- }
- else if (str[idx] != ',') {
- raise_errmsg("Expecting , delimiter", pystr, idx);
- goto bail;
- }
- idx++;
-
- /* skip whitespace after , */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
- }
- }
-
- /* verify that idx < end_idx, str[idx] should be ']' */
- if (idx > end_idx || str[idx] != ']') {
- raise_errmsg("Expecting object", pystr, end_idx);
- goto bail;
- }
- *next_idx_ptr = idx + 1;
- return rval;
-bail:
- Py_XDECREF(val);
- Py_DECREF(rval);
- return NULL;
-}
-
-static PyObject *
-_parse_array_unicode(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr) {
- /* Read a JSON array from PyString pystr.
- idx is the index of the first character after the opening brace.
- *next_idx_ptr is a return-by-reference index to the first character after
- the closing brace.
-
- Returns a new PyList
- */
- Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr);
- Py_ssize_t end_idx = PyUnicode_GET_SIZE(pystr) - 1;
- PyObject *val = NULL;
- PyObject *rval = PyList_New(0);
- Py_ssize_t next_idx;
- if (rval == NULL)
- return NULL;
-
- /* skip whitespace after [ */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* only loop if the array is non-empty */
- if (idx <= end_idx && str[idx] != ']') {
- while (idx <= end_idx) {
-
- /* read any JSON term */
- val = scan_once_unicode(s, pystr, idx, &next_idx);
- if (val == NULL)
- goto bail;
-
- if (PyList_Append(rval, val) == -1)
- goto bail;
-
- Py_CLEAR(val);
- idx = next_idx;
-
- /* skip whitespace between term and , */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
-
- /* bail if the array is closed or we didn't get the , delimiter */
- if (idx > end_idx) break;
- if (str[idx] == ']') {
- break;
- }
- else if (str[idx] != ',') {
- raise_errmsg("Expecting , delimiter", pystr, idx);
- goto bail;
- }
- idx++;
-
- /* skip whitespace after , */
- while (idx <= end_idx && IS_WHITESPACE(str[idx])) idx++;
- }
- }
-
- /* verify that idx < end_idx, str[idx] should be ']' */
- if (idx > end_idx || str[idx] != ']') {
- raise_errmsg("Expecting object", pystr, end_idx);
- goto bail;
- }
- *next_idx_ptr = idx + 1;
- return rval;
-bail:
- Py_XDECREF(val);
- Py_DECREF(rval);
- return NULL;
-}
-
-static PyObject *
-_parse_constant(PyScannerObject *s, char *constant, Py_ssize_t idx, Py_ssize_t *next_idx_ptr) {
- /* Read a JSON constant from PyString pystr.
- constant is the constant string that was found
- ("NaN", "Infinity", "-Infinity").
- idx is the index of the first character of the constant
- *next_idx_ptr is a return-by-reference index to the first character after
- the constant.
-
- Returns the result of parse_constant
- */
- PyObject *cstr;
- PyObject *rval;
- /* constant is "NaN", "Infinity", or "-Infinity" */
- cstr = PyString_InternFromString(constant);
- if (cstr == NULL)
- return NULL;
-
- /* rval = parse_constant(constant) */
- rval = PyObject_CallFunctionObjArgs(s->parse_constant, cstr, NULL);
- idx += PyString_GET_SIZE(cstr);
- Py_DECREF(cstr);
- *next_idx_ptr = idx;
- return rval;
-}
-
-static PyObject *
-_match_number_str(PyScannerObject *s, PyObject *pystr, Py_ssize_t start, Py_ssize_t *next_idx_ptr) {
- /* Read a JSON number from PyString pystr.
- idx is the index of the first character of the number
- *next_idx_ptr is a return-by-reference index to the first character after
- the number.
-
- Returns a new PyObject representation of that number:
- PyInt, PyLong, or PyFloat.
- May return other types if parse_int or parse_float are set
- */
- char *str = PyString_AS_STRING(pystr);
- Py_ssize_t end_idx = PyString_GET_SIZE(pystr) - 1;
- Py_ssize_t idx = start;
- int is_float = 0;
- PyObject *rval;
- PyObject *numstr;
-
- /* read a sign if it's there, make sure it's not the end of the string */
- if (str[idx] == '-') {
- idx++;
- if (idx > end_idx) {
- PyErr_SetNone(PyExc_StopIteration);
- return NULL;
- }
- }
-
- /* read as many integer digits as we find as long as it doesn't start with 0 */
- if (str[idx] >= '1' && str[idx] <= '9') {
- idx++;
- while (idx <= end_idx && str[idx] >= '0' && str[idx] <= '9') idx++;
- }
- /* if it starts with 0 we only expect one integer digit */
- else if (str[idx] == '0') {
- idx++;
- }
- /* no integer digits, error */
- else {
- PyErr_SetNone(PyExc_StopIteration);
- return NULL;
- }
-
- /* if the next char is '.' followed by a digit then read all float digits */
- if (idx < end_idx && str[idx] == '.' && str[idx + 1] >= '0' && str[idx + 1] <= '9') {
- is_float = 1;
- idx += 2;
- while (idx <= end_idx && str[idx] >= '0' && str[idx] <= '9') idx++;
- }
-
- /* if the next char is 'e' or 'E' then maybe read the exponent (or backtrack) */
- if (idx < end_idx && (str[idx] == 'e' || str[idx] == 'E')) {
-
- /* save the index of the 'e' or 'E' just in case we need to backtrack */
- Py_ssize_t e_start = idx;
- idx++;
-
- /* read an exponent sign if present */
- if (idx < end_idx && (str[idx] == '-' || str[idx] == '+')) idx++;
-
- /* read all digits */
- while (idx <= end_idx && str[idx] >= '0' && str[idx] <= '9') idx++;
-
- /* if we got a digit, then parse as float. if not, backtrack */
- if (str[idx - 1] >= '0' && str[idx - 1] <= '9') {
- is_float = 1;
- }
- else {
- idx = e_start;
- }
- }
-
- /* copy the section we determined to be a number */
- numstr = PyString_FromStringAndSize(&str[start], idx - start);
- if (numstr == NULL)
- return NULL;
- if (is_float) {
- /* parse as a float using a fast path if available, otherwise call user defined method */
- if (s->parse_float != (PyObject *)&PyFloat_Type) {
- rval = PyObject_CallFunctionObjArgs(s->parse_float, numstr, NULL);
- }
- else {
- rval = PyFloat_FromDouble(PyOS_ascii_atof(PyString_AS_STRING(numstr)));
- }
- }
- else {
- /* parse as an int using a fast path if available, otherwise call user defined method */
- if (s->parse_int != (PyObject *)&PyInt_Type) {
- rval = PyObject_CallFunctionObjArgs(s->parse_int, numstr, NULL);
- }
- else {
- rval = PyInt_FromString(PyString_AS_STRING(numstr), NULL, 10);
- }
- }
- Py_DECREF(numstr);
- *next_idx_ptr = idx;
- return rval;
-}
-
-static PyObject *
-_match_number_unicode(PyScannerObject *s, PyObject *pystr, Py_ssize_t start, Py_ssize_t *next_idx_ptr) {
- /* Read a JSON number from PyUnicode pystr.
- idx is the index of the first character of the number
- *next_idx_ptr is a return-by-reference index to the first character after
- the number.
-
- Returns a new PyObject representation of that number:
- PyInt, PyLong, or PyFloat.
- May return other types if parse_int or parse_float are set
- */
- Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr);
- Py_ssize_t end_idx = PyUnicode_GET_SIZE(pystr) - 1;
- Py_ssize_t idx = start;
- int is_float = 0;
- PyObject *rval;
- PyObject *numstr;
-
- /* read a sign if it's there, make sure it's not the end of the string */
- if (str[idx] == '-') {
- idx++;
- if (idx > end_idx) {
- PyErr_SetNone(PyExc_StopIteration);
- return NULL;
- }
- }
-
- /* read as many integer digits as we find as long as it doesn't start with 0 */
- if (str[idx] >= '1' && str[idx] <= '9') {
- idx++;
- while (idx <= end_idx && str[idx] >= '0' && str[idx] <= '9') idx++;
- }
- /* if it starts with 0 we only expect one integer digit */
- else if (str[idx] == '0') {
- idx++;
- }
- /* no integer digits, error */
- else {
- PyErr_SetNone(PyExc_StopIteration);
- return NULL;
- }
-
- /* if the next char is '.' followed by a digit then read all float digits */
- if (idx < end_idx && str[idx] == '.' && str[idx + 1] >= '0' && str[idx + 1] <= '9') {
- is_float = 1;
- idx += 2;
- while (idx < end_idx && str[idx] >= '0' && str[idx] <= '9') idx++;
- }
-
- /* if the next char is 'e' or 'E' then maybe read the exponent (or backtrack) */
- if (idx < end_idx && (str[idx] == 'e' || str[idx] == 'E')) {
- Py_ssize_t e_start = idx;
- idx++;
-
- /* read an exponent sign if present */
- if (idx < end_idx && (str[idx] == '-' || str[idx] == '+')) idx++;
-
- /* read all digits */
- while (idx <= end_idx && str[idx] >= '0' && str[idx] <= '9') idx++;
-
- /* if we got a digit, then parse as float. if not, backtrack */
- if (str[idx - 1] >= '0' && str[idx - 1] <= '9') {
- is_float = 1;
- }
- else {
- idx = e_start;
- }
- }
-
- /* copy the section we determined to be a number */
- numstr = PyUnicode_FromUnicode(&str[start], idx - start);
- if (numstr == NULL)
- return NULL;
- if (is_float) {
- /* parse as a float using a fast path if available, otherwise call user defined method */
- if (s->parse_float != (PyObject *)&PyFloat_Type) {
- rval = PyObject_CallFunctionObjArgs(s->parse_float, numstr, NULL);
- }
- else {
- rval = PyFloat_FromString(numstr, NULL);
- }
- }
- else {
- /* no fast path for unicode -> int, just call */
- rval = PyObject_CallFunctionObjArgs(s->parse_int, numstr, NULL);
- }
- Py_DECREF(numstr);
- *next_idx_ptr = idx;
- return rval;
-}
-
-static PyObject *
-scan_once_str(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr)
-{
- /* Read one JSON term (of any kind) from PyString pystr.
- idx is the index of the first character of the term
- *next_idx_ptr is a return-by-reference index to the first character after
- the number.
-
- Returns a new PyObject representation of the term.
- */
- char *str = PyString_AS_STRING(pystr);
- Py_ssize_t length = PyString_GET_SIZE(pystr);
- if (idx >= length) {
- PyErr_SetNone(PyExc_StopIteration);
- return NULL;
- }
- switch (str[idx]) {
- case '"':
- /* string */
- return scanstring_str(pystr, idx + 1,
- PyString_AS_STRING(s->encoding),
- PyObject_IsTrue(s->strict),
- next_idx_ptr);
- case '{':
- /* object */
- return _parse_object_str(s, pystr, idx + 1, next_idx_ptr);
- case '[':
- /* array */
- return _parse_array_str(s, pystr, idx + 1, next_idx_ptr);
- case 'n':
- /* null */
- if ((idx + 3 < length) && str[idx + 1] == 'u' && str[idx + 2] == 'l' && str[idx + 3] == 'l') {
- Py_INCREF(Py_None);
- *next_idx_ptr = idx + 4;
- return Py_None;
- }
- break;
- case 't':
- /* true */
- if ((idx + 3 < length) && str[idx + 1] == 'r' && str[idx + 2] == 'u' && str[idx + 3] == 'e') {
- Py_INCREF(Py_True);
- *next_idx_ptr = idx + 4;
- return Py_True;
- }
- break;
- case 'f':
- /* false */
- if ((idx + 4 < length) && str[idx + 1] == 'a' && str[idx + 2] == 'l' && str[idx + 3] == 's' && str[idx + 4] == 'e') {
- Py_INCREF(Py_False);
- *next_idx_ptr = idx + 5;
- return Py_False;
- }
- break;
- case 'N':
- /* NaN */
- if ((idx + 2 < length) && str[idx + 1] == 'a' && str[idx + 2] == 'N') {
- return _parse_constant(s, "NaN", idx, next_idx_ptr);
- }
- break;
- case 'I':
- /* Infinity */
- if ((idx + 7 < length) && str[idx + 1] == 'n' && str[idx + 2] == 'f' && str[idx + 3] == 'i' && str[idx + 4] == 'n' && str[idx + 5] == 'i' && str[idx + 6] == 't' && str[idx + 7] == 'y') {
- return _parse_constant(s, "Infinity", idx, next_idx_ptr);
- }
- break;
- case '-':
- /* -Infinity */
- if ((idx + 8 < length) && str[idx + 1] == 'I' && str[idx + 2] == 'n' && str[idx + 3] == 'f' && str[idx + 4] == 'i' && str[idx + 5] == 'n' && str[idx + 6] == 'i' && str[idx + 7] == 't' && str[idx + 8] == 'y') {
- return _parse_constant(s, "-Infinity", idx, next_idx_ptr);
- }
- break;
- }
- /* Didn't find a string, object, array, or named constant. Look for a number. */
- return _match_number_str(s, pystr, idx, next_idx_ptr);
-}
-
-static PyObject *
-scan_once_unicode(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ssize_t *next_idx_ptr)
-{
- /* Read one JSON term (of any kind) from PyUnicode pystr.
- idx is the index of the first character of the term
- *next_idx_ptr is a return-by-reference index to the first character after
- the number.
-
- Returns a new PyObject representation of the term.
- */
- Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr);
- Py_ssize_t length = PyUnicode_GET_SIZE(pystr);
- if (idx >= length) {
- PyErr_SetNone(PyExc_StopIteration);
- return NULL;
- }
- switch (str[idx]) {
- case '"':
- /* string */
- return scanstring_unicode(pystr, idx + 1,
- PyObject_IsTrue(s->strict),
- next_idx_ptr);
- case '{':
- /* object */
- return _parse_object_unicode(s, pystr, idx + 1, next_idx_ptr);
- case '[':
- /* array */
- return _parse_array_unicode(s, pystr, idx + 1, next_idx_ptr);
- case 'n':
- /* null */
- if ((idx + 3 < length) && str[idx + 1] == 'u' && str[idx + 2] == 'l' && str[idx + 3] == 'l') {
- Py_INCREF(Py_None);
- *next_idx_ptr = idx + 4;
- return Py_None;
- }
- break;
- case 't':
- /* true */
- if ((idx + 3 < length) && str[idx + 1] == 'r' && str[idx + 2] == 'u' && str[idx + 3] == 'e') {
- Py_INCREF(Py_True);
- *next_idx_ptr = idx + 4;
- return Py_True;
- }
- break;
- case 'f':
- /* false */
- if ((idx + 4 < length) && str[idx + 1] == 'a' && str[idx + 2] == 'l' && str[idx + 3] == 's' && str[idx + 4] == 'e') {
- Py_INCREF(Py_False);
- *next_idx_ptr = idx + 5;
- return Py_False;
- }
- break;
- case 'N':
- /* NaN */
- if ((idx + 2 < length) && str[idx + 1] == 'a' && str[idx + 2] == 'N') {
- return _parse_constant(s, "NaN", idx, next_idx_ptr);
- }
- break;
- case 'I':
- /* Infinity */
- if ((idx + 7 < length) && str[idx + 1] == 'n' && str[idx + 2] == 'f' && str[idx + 3] == 'i' && str[idx + 4] == 'n' && str[idx + 5] == 'i' && str[idx + 6] == 't' && str[idx + 7] == 'y') {
- return _parse_constant(s, "Infinity", idx, next_idx_ptr);
- }
- break;
- case '-':
- /* -Infinity */
- if ((idx + 8 < length) && str[idx + 1] == 'I' && str[idx + 2] == 'n' && str[idx + 3] == 'f' && str[idx + 4] == 'i' && str[idx + 5] == 'n' && str[idx + 6] == 'i' && str[idx + 7] == 't' && str[idx + 8] == 'y') {
- return _parse_constant(s, "-Infinity", idx, next_idx_ptr);
- }
- break;
- }
- /* Didn't find a string, object, array, or named constant. Look for a number. */
- return _match_number_unicode(s, pystr, idx, next_idx_ptr);
-}
-
-static PyObject *
-scanner_call(PyObject *self, PyObject *args, PyObject *kwds)
-{
- /* Python callable interface to scan_once_{str,unicode} */
- PyObject *pystr;
- PyObject *rval;
- Py_ssize_t idx;
- Py_ssize_t next_idx = -1;
- static char *kwlist[] = {"string", "idx", NULL};
- PyScannerObject *s;
- assert(PyScanner_Check(self));
- s = (PyScannerObject *)self;
- if (!PyArg_ParseTupleAndKeywords(args, kwds, "OO&:scan_once", kwlist, &pystr, _convertPyInt_AsSsize_t, &idx))
- return NULL;
-
- if (PyString_Check(pystr)) {
- rval = scan_once_str(s, pystr, idx, &next_idx);
- }
- else if (PyUnicode_Check(pystr)) {
- rval = scan_once_unicode(s, pystr, idx, &next_idx);
- }
- else {
- PyErr_Format(PyExc_TypeError,
- "first argument must be a string, not %.80s",
- Py_TYPE(pystr)->tp_name);
- return NULL;
- }
- return _build_rval_index_tuple(rval, next_idx);
-}
-
-static int
-scanner_init(PyObject *self, PyObject *args, PyObject *kwds)
-{
- /* Initialize Scanner object */
- PyObject *ctx;
- static char *kwlist[] = {"context", NULL};
- PyScannerObject *s;
-
- assert(PyScanner_Check(self));
- s = (PyScannerObject *)self;
-
- if (!PyArg_ParseTupleAndKeywords(args, kwds, "O:make_scanner", kwlist, &ctx))
- return -1;
-
- s->encoding = NULL;
- s->strict = NULL;
- s->object_hook = NULL;
- s->parse_float = NULL;
- s->parse_int = NULL;
- s->parse_constant = NULL;
-
- /* PyString_AS_STRING is used on encoding */
- s->encoding = PyObject_GetAttrString(ctx, "encoding");
- if (s->encoding == Py_None) {
- Py_DECREF(Py_None);
- s->encoding = PyString_InternFromString(DEFAULT_ENCODING);
- }
- else if (PyUnicode_Check(s->encoding)) {
- PyObject *tmp = PyUnicode_AsEncodedString(s->encoding, NULL, NULL);
- Py_DECREF(s->encoding);
- s->encoding = tmp;
- }
- if (s->encoding == NULL || !PyString_Check(s->encoding))
- goto bail;
-
- /* All of these will fail "gracefully" so we don't need to verify them */
- s->strict = PyObject_GetAttrString(ctx, "strict");
- if (s->strict == NULL)
- goto bail;
- s->object_hook = PyObject_GetAttrString(ctx, "object_hook");
- if (s->object_hook == NULL)
- goto bail;
- s->parse_float = PyObject_GetAttrString(ctx, "parse_float");
- if (s->parse_float == NULL)
- goto bail;
- s->parse_int = PyObject_GetAttrString(ctx, "parse_int");
- if (s->parse_int == NULL)
- goto bail;
- s->parse_constant = PyObject_GetAttrString(ctx, "parse_constant");
- if (s->parse_constant == NULL)
- goto bail;
-
- return 0;
-
-bail:
- Py_CLEAR(s->encoding);
- Py_CLEAR(s->strict);
- Py_CLEAR(s->object_hook);
- Py_CLEAR(s->parse_float);
- Py_CLEAR(s->parse_int);
- Py_CLEAR(s->parse_constant);
- return -1;
-}
-
-PyDoc_STRVAR(scanner_doc, "JSON scanner object");
-
-static
-PyTypeObject PyScannerType = {
- PyObject_HEAD_INIT(0)
- 0, /* tp_internal */
- "Scanner", /* tp_name */
- sizeof(PyScannerObject), /* tp_basicsize */
- 0, /* tp_itemsize */
- scanner_dealloc, /* tp_dealloc */
- 0, /* tp_print */
- 0, /* tp_getattr */
- 0, /* tp_setattr */
- 0, /* tp_compare */
- 0, /* tp_repr */
- 0, /* tp_as_number */
- 0, /* tp_as_sequence */
- 0, /* tp_as_mapping */
- 0, /* tp_hash */
- scanner_call, /* tp_call */
- 0, /* tp_str */
- 0,/* PyObject_GenericGetAttr, */ /* tp_getattro */
- 0,/* PyObject_GenericSetAttr, */ /* tp_setattro */
- 0, /* tp_as_buffer */
- Py_TPFLAGS_DEFAULT, /* tp_flags */
- scanner_doc, /* tp_doc */
- 0, /* tp_traverse */
- 0, /* tp_clear */
- 0, /* tp_richcompare */
- 0, /* tp_weaklistoffset */
- 0, /* tp_iter */
- 0, /* tp_iternext */
- 0, /* tp_methods */
- scanner_members, /* tp_members */
- 0, /* tp_getset */
- 0, /* tp_base */
- 0, /* tp_dict */
- 0, /* tp_descr_get */
- 0, /* tp_descr_set */
- 0, /* tp_dictoffset */
- scanner_init, /* tp_init */
- 0,/* PyType_GenericAlloc, */ /* tp_alloc */
- 0,/* PyType_GenericNew, */ /* tp_new */
- 0,/* _PyObject_Del, */ /* tp_free */
-};
-
-static int
-encoder_init(PyObject *self, PyObject *args, PyObject *kwds)
-{
- /* initialize Encoder object */
- static char *kwlist[] = {"markers", "default", "encoder", "indent", "key_separator", "item_separator", "sort_keys", "skipkeys", "allow_nan", NULL};
-
- PyEncoderObject *s;
- PyObject *allow_nan;
-
- assert(PyEncoder_Check(self));
- s = (PyEncoderObject *)self;
-
- s->markers = NULL;
- s->defaultfn = NULL;
- s->encoder = NULL;
- s->indent = NULL;
- s->key_separator = NULL;
- s->item_separator = NULL;
- s->sort_keys = NULL;
- s->skipkeys = NULL;
-
- if (!PyArg_ParseTupleAndKeywords(args, kwds, "OOOOOOOOO:make_encoder", kwlist,
- &s->markers, &s->defaultfn, &s->encoder, &s->indent, &s->key_separator, &s->item_separator, &s->sort_keys, &s->skipkeys, &allow_nan))
- return -1;
-
- Py_INCREF(s->markers);
- Py_INCREF(s->defaultfn);
- Py_INCREF(s->encoder);
- Py_INCREF(s->indent);
- Py_INCREF(s->key_separator);
- Py_INCREF(s->item_separator);
- Py_INCREF(s->sort_keys);
- Py_INCREF(s->skipkeys);
- s->fast_encode = (PyCFunction_Check(s->encoder) && PyCFunction_GetFunction(s->encoder) == (PyCFunction)py_encode_basestring_ascii);
- s->allow_nan = PyObject_IsTrue(allow_nan);
- return 0;
-}
-
-static PyObject *
-encoder_call(PyObject *self, PyObject *args, PyObject *kwds)
-{
- /* Python callable interface to encode_listencode_obj */
- static char *kwlist[] = {"obj", "_current_indent_level", NULL};
- PyObject *obj;
- PyObject *rval;
- Py_ssize_t indent_level;
- PyEncoderObject *s;
- assert(PyEncoder_Check(self));
- s = (PyEncoderObject *)self;
- if (!PyArg_ParseTupleAndKeywords(args, kwds, "OO&:_iterencode", kwlist,
- &obj, _convertPyInt_AsSsize_t, &indent_level))
- return NULL;
- rval = PyList_New(0);
- if (rval == NULL)
- return NULL;
- if (encoder_listencode_obj(s, rval, obj, indent_level)) {
- Py_DECREF(rval);
- return NULL;
- }
- return rval;
-}
-
-static PyObject *
-_encoded_const(PyObject *obj)
-{
- /* Return the JSON string representation of None, True, False */
- if (obj == Py_None) {
- static PyObject *s_null = NULL;
- if (s_null == NULL) {
- s_null = PyString_InternFromString("null");
- }
- Py_INCREF(s_null);
- return s_null;
- }
- else if (obj == Py_True) {
- static PyObject *s_true = NULL;
- if (s_true == NULL) {
- s_true = PyString_InternFromString("true");
- }
- Py_INCREF(s_true);
- return s_true;
- }
- else if (obj == Py_False) {
- static PyObject *s_false = NULL;
- if (s_false == NULL) {
- s_false = PyString_InternFromString("false");
- }
- Py_INCREF(s_false);
- return s_false;
- }
- else {
- PyErr_SetString(PyExc_ValueError, "not a const");
- return NULL;
- }
-}
-
-static PyObject *
-encoder_encode_float(PyEncoderObject *s, PyObject *obj)
-{
- /* Return the JSON representation of a PyFloat */
- double i = PyFloat_AS_DOUBLE(obj);
- if (!Py_IS_FINITE(i)) {
- if (!s->allow_nan) {
- PyErr_SetString(PyExc_ValueError, "Out of range float values are not JSON compliant");
- return NULL;
- }
- if (i > 0) {
- return PyString_FromString("Infinity");
- }
- else if (i < 0) {
- return PyString_FromString("-Infinity");
- }
- else {
- return PyString_FromString("NaN");
- }
- }
- /* Use a better float format here? */
- return PyObject_Repr(obj);
-}
-
-static PyObject *
-encoder_encode_string(PyEncoderObject *s, PyObject *obj)
-{
- /* Return the JSON representation of a string */
- if (s->fast_encode)
- return py_encode_basestring_ascii(NULL, obj);
- else
- return PyObject_CallFunctionObjArgs(s->encoder, obj, NULL);
-}
-
-static int
-_steal_list_append(PyObject *lst, PyObject *stolen)
-{
- /* Append stolen and then decrement its reference count */
- int rval = PyList_Append(lst, stolen);
- Py_DECREF(stolen);
- return rval;
-}
-
-static int
-encoder_listencode_obj(PyEncoderObject *s, PyObject *rval, PyObject *obj, Py_ssize_t indent_level)
-{
- /* Encode Python object obj to a JSON term, rval is a PyList */
- PyObject *newobj;
- int rv;
-
- if (obj == Py_None || obj == Py_True || obj == Py_False) {
- PyObject *cstr = _encoded_const(obj);
- if (cstr == NULL)
- return -1;
- return _steal_list_append(rval, cstr);
- }
- else if (PyString_Check(obj) || PyUnicode_Check(obj))
- {
- PyObject *encoded = encoder_encode_string(s, obj);
- if (encoded == NULL)
- return -1;
- return _steal_list_append(rval, encoded);
- }
- else if (PyInt_Check(obj) || PyLong_Check(obj)) {
- PyObject *encoded = PyObject_Str(obj);
- if (encoded == NULL)
- return -1;
- return _steal_list_append(rval, encoded);
- }
- else if (PyFloat_Check(obj)) {
- PyObject *encoded = encoder_encode_float(s, obj);
- if (encoded == NULL)
- return -1;
- return _steal_list_append(rval, encoded);
- }
- else if (PyList_Check(obj) || PyTuple_Check(obj)) {
- return encoder_listencode_list(s, rval, obj, indent_level);
- }
- else if (PyDict_Check(obj)) {
- return encoder_listencode_dict(s, rval, obj, indent_level);
- }
- else {
- PyObject *ident = NULL;
- if (s->markers != Py_None) {
- int has_key;
- ident = PyLong_FromVoidPtr(obj);
- if (ident == NULL)
- return -1;
- has_key = PyDict_Contains(s->markers, ident);
- if (has_key) {
- if (has_key != -1)
- PyErr_SetString(PyExc_ValueError, "Circular reference detected");
- Py_DECREF(ident);
- return -1;
- }
- if (PyDict_SetItem(s->markers, ident, obj)) {
- Py_DECREF(ident);
- return -1;
- }
- }
- newobj = PyObject_CallFunctionObjArgs(s->defaultfn, obj, NULL);
- if (newobj == NULL) {
- Py_XDECREF(ident);
- return -1;
- }
- rv = encoder_listencode_obj(s, rval, newobj, indent_level);
- Py_DECREF(newobj);
- if (rv) {
- Py_XDECREF(ident);
- return -1;
- }
- if (ident != NULL) {
- if (PyDict_DelItem(s->markers, ident)) {
- Py_XDECREF(ident);
- return -1;
- }
- Py_XDECREF(ident);
- }
- return rv;
- }
-}
-
-static int
-encoder_listencode_dict(PyEncoderObject *s, PyObject *rval, PyObject *dct, Py_ssize_t indent_level)
-{
- /* Encode Python dict dct a JSON term, rval is a PyList */
- static PyObject *open_dict = NULL;
- static PyObject *close_dict = NULL;
- static PyObject *empty_dict = NULL;
- PyObject *kstr = NULL;
- PyObject *ident = NULL;
- PyObject *key, *value;
- Py_ssize_t pos;
- int skipkeys;
- Py_ssize_t idx;
-
- if (open_dict == NULL || close_dict == NULL || empty_dict == NULL) {
- open_dict = PyString_InternFromString("{");
- close_dict = PyString_InternFromString("}");
- empty_dict = PyString_InternFromString("{}");
- if (open_dict == NULL || close_dict == NULL || empty_dict == NULL)
- return -1;
- }
- if (PyDict_Size(dct) == 0)
- return PyList_Append(rval, empty_dict);
-
- if (s->markers != Py_None) {
- int has_key;
- ident = PyLong_FromVoidPtr(dct);
- if (ident == NULL)
- goto bail;
- has_key = PyDict_Contains(s->markers, ident);
- if (has_key) {
- if (has_key != -1)
- PyErr_SetString(PyExc_ValueError, "Circular reference detected");
- goto bail;
- }
- if (PyDict_SetItem(s->markers, ident, dct)) {
- goto bail;
- }
- }
-
- if (PyList_Append(rval, open_dict))
- goto bail;
-
- if (s->indent != Py_None) {
- /* TODO: DOES NOT RUN */
- indent_level += 1;
- /*
- newline_indent = '\n' + (' ' * (_indent * _current_indent_level))
- separator = _item_separator + newline_indent
- buf += newline_indent
- */
- }
-
- /* TODO: C speedup not implemented for sort_keys */
-
- pos = 0;
- skipkeys = PyObject_IsTrue(s->skipkeys);
- idx = 0;
- while (PyDict_Next(dct, &pos, &key, &value)) {
- PyObject *encoded;
-
- if (PyString_Check(key) || PyUnicode_Check(key)) {
- Py_INCREF(key);
- kstr = key;
- }
- else if (PyFloat_Check(key)) {
- kstr = encoder_encode_float(s, key);
- if (kstr == NULL)
- goto bail;
- }
- else if (PyInt_Check(key) || PyLong_Check(key)) {
- kstr = PyObject_Str(key);
- if (kstr == NULL)
- goto bail;
- }
- else if (key == Py_True || key == Py_False || key == Py_None) {
- kstr = _encoded_const(key);
- if (kstr == NULL)
- goto bail;
- }
- else if (skipkeys) {
- continue;
- }
- else {
- /* TODO: include repr of key */
- PyErr_SetString(PyExc_ValueError, "keys must be a string");
- goto bail;
- }
-
- if (idx) {
- if (PyList_Append(rval, s->item_separator))
- goto bail;
- }
-
- encoded = encoder_encode_string(s, kstr);
- Py_CLEAR(kstr);
- if (encoded == NULL)
- goto bail;
- if (PyList_Append(rval, encoded)) {
- Py_DECREF(encoded);
- goto bail;
- }
- Py_DECREF(encoded);
- if (PyList_Append(rval, s->key_separator))
- goto bail;
- if (encoder_listencode_obj(s, rval, value, indent_level))
- goto bail;
- idx += 1;
- }
- if (ident != NULL) {
- if (PyDict_DelItem(s->markers, ident))
- goto bail;
- Py_CLEAR(ident);
- }
- if (s->indent != Py_None) {
- /* TODO: DOES NOT RUN */
- indent_level -= 1;
- /*
- yield '\n' + (' ' * (_indent * _current_indent_level))
- */
- }
- if (PyList_Append(rval, close_dict))
- goto bail;
- return 0;
-
-bail:
- Py_XDECREF(kstr);
- Py_XDECREF(ident);
- return -1;
-}
-
-
-static int
-encoder_listencode_list(PyEncoderObject *s, PyObject *rval, PyObject *seq, Py_ssize_t indent_level)
-{
- /* Encode Python list seq to a JSON term, rval is a PyList */
- static PyObject *open_array = NULL;
- static PyObject *close_array = NULL;
- static PyObject *empty_array = NULL;
- PyObject *ident = NULL;
- PyObject *s_fast = NULL;
- Py_ssize_t num_items;
- PyObject **seq_items;
- Py_ssize_t i;
-
- if (open_array == NULL || close_array == NULL || empty_array == NULL) {
- open_array = PyString_InternFromString("[");
- close_array = PyString_InternFromString("]");
- empty_array = PyString_InternFromString("[]");
- if (open_array == NULL || close_array == NULL || empty_array == NULL)
- return -1;
- }
- ident = NULL;
- s_fast = PySequence_Fast(seq, "_iterencode_list needs a sequence");
- if (s_fast == NULL)
- return -1;
- num_items = PySequence_Fast_GET_SIZE(s_fast);
- if (num_items == 0) {
- Py_DECREF(s_fast);
- return PyList_Append(rval, empty_array);
- }
-
- if (s->markers != Py_None) {
- int has_key;
- ident = PyLong_FromVoidPtr(seq);
- if (ident == NULL)
- goto bail;
- has_key = PyDict_Contains(s->markers, ident);
- if (has_key) {
- if (has_key != -1)
- PyErr_SetString(PyExc_ValueError, "Circular reference detected");
- goto bail;
- }
- if (PyDict_SetItem(s->markers, ident, seq)) {
- goto bail;
- }
- }
-
- seq_items = PySequence_Fast_ITEMS(s_fast);
- if (PyList_Append(rval, open_array))
- goto bail;
- if (s->indent != Py_None) {
- /* TODO: DOES NOT RUN */
- indent_level += 1;
- /*
- newline_indent = '\n' + (' ' * (_indent * _current_indent_level))
- separator = _item_separator + newline_indent
- buf += newline_indent
- */
- }
- for (i = 0; i < num_items; i++) {
- PyObject *obj = seq_items[i];
- if (i) {
- if (PyList_Append(rval, s->item_separator))
- goto bail;
- }
- if (encoder_listencode_obj(s, rval, obj, indent_level))
- goto bail;
- }
- if (ident != NULL) {
- if (PyDict_DelItem(s->markers, ident))
- goto bail;
- Py_CLEAR(ident);
- }
- if (s->indent != Py_None) {
- /* TODO: DOES NOT RUN */
- indent_level -= 1;
- /*
- yield '\n' + (' ' * (_indent * _current_indent_level))
- */
- }
- if (PyList_Append(rval, close_array))
- goto bail;
- Py_DECREF(s_fast);
- return 0;
-
-bail:
- Py_XDECREF(ident);
- Py_DECREF(s_fast);
- return -1;
-}
-
-static void
-encoder_dealloc(PyObject *self)
-{
- /* Deallocate Encoder */
- PyEncoderObject *s;
- assert(PyEncoder_Check(self));
- s = (PyEncoderObject *)self;
- Py_CLEAR(s->markers);
- Py_CLEAR(s->defaultfn);
- Py_CLEAR(s->encoder);
- Py_CLEAR(s->indent);
- Py_CLEAR(s->key_separator);
- Py_CLEAR(s->item_separator);
- Py_CLEAR(s->sort_keys);
- Py_CLEAR(s->skipkeys);
- self->ob_type->tp_free(self);
-}
-
-PyDoc_STRVAR(encoder_doc, "_iterencode(obj, _current_indent_level) -> iterable");
-
-static
-PyTypeObject PyEncoderType = {
- PyObject_HEAD_INIT(0)
- 0, /* tp_internal */
- "Encoder", /* tp_name */
- sizeof(PyEncoderObject), /* tp_basicsize */
- 0, /* tp_itemsize */
- encoder_dealloc, /* tp_dealloc */
- 0, /* tp_print */
- 0, /* tp_getattr */
- 0, /* tp_setattr */
- 0, /* tp_compare */
- 0, /* tp_repr */
- 0, /* tp_as_number */
- 0, /* tp_as_sequence */
- 0, /* tp_as_mapping */
- 0, /* tp_hash */
- encoder_call, /* tp_call */
- 0, /* tp_str */
- 0,/* PyObject_GenericGetAttr, */ /* tp_getattro */
- 0,/* PyObject_GenericSetAttr, */ /* tp_setattro */
- 0, /* tp_as_buffer */
- Py_TPFLAGS_DEFAULT, /* tp_flags */
- encoder_doc, /* tp_doc */
- 0, /* tp_traverse */
- 0, /* tp_clear */
- 0, /* tp_richcompare */
- 0, /* tp_weaklistoffset */
- 0, /* tp_iter */
- 0, /* tp_iternext */
- 0, /* tp_methods */
- encoder_members, /* tp_members */
- 0, /* tp_getset */
- 0, /* tp_base */
- 0, /* tp_dict */
- 0, /* tp_descr_get */
- 0, /* tp_descr_set */
- 0, /* tp_dictoffset */
- encoder_init, /* tp_init */
- 0,/* PyType_GenericAlloc, */ /* tp_alloc */
- 0,/* PyType_GenericNew, */ /* tp_new */
- 0,/* _PyObject_Del, */ /* tp_free */
-};
-
-static PyMethodDef speedups_methods[] = {
- {"encode_basestring_ascii",
- (PyCFunction)py_encode_basestring_ascii,
- METH_O,
- pydoc_encode_basestring_ascii},
- {"scanstring",
- (PyCFunction)py_scanstring,
- METH_VARARGS,
- pydoc_scanstring},
- {NULL, NULL, 0, NULL}
-};
-
-PyDoc_STRVAR(module_doc,
-"simplejson speedups\n");
-
-void
-init_speedups(void)
-{
- PyObject *m;
- PyScannerType.tp_getattro = PyObject_GenericGetAttr;
- PyScannerType.tp_setattro = PyObject_GenericSetAttr;
- PyScannerType.tp_alloc = PyType_GenericAlloc;
- PyScannerType.tp_new = PyType_GenericNew;
- PyScannerType.tp_free = _PyObject_Del;
- if (PyType_Ready(&PyScannerType) < 0)
- return;
- PyEncoderType.tp_getattro = PyObject_GenericGetAttr;
- PyEncoderType.tp_setattro = PyObject_GenericSetAttr;
- PyEncoderType.tp_alloc = PyType_GenericAlloc;
- PyEncoderType.tp_new = PyType_GenericNew;
- PyEncoderType.tp_free = _PyObject_Del;
- if (PyType_Ready(&PyEncoderType) < 0)
- return;
- m = Py_InitModule3("_speedups", speedups_methods, module_doc);
- Py_INCREF((PyObject*)&PyScannerType);
- PyModule_AddObject(m, "make_scanner", (PyObject*)&PyScannerType);
- Py_INCREF((PyObject*)&PyEncoderType);
- PyModule_AddObject(m, "make_encoder", (PyObject*)&PyEncoderType);
-}
View
348 simplejson/decoder.py
@@ -1,348 +0,0 @@
-"""Implementation of JSONDecoder
-"""
-import re
-import sys
-import struct
-
-from simplejson.scanner import make_scanner
-try:
- from simplejson._speedups import scanstring as c_scanstring
-except ImportError:
- c_scanstring = None
-
-__all__ = ['JSONDecoder']
-
-FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
-
-def _floatconstants():
- _BYTES = '7FF80000000000007FF0000000000000'.decode('hex')
- if sys.byteorder != 'big':
- _BYTES = _BYTES[:8][::-1] + _BYTES[8:][::-1]
- nan, inf = struct.unpack('dd', _BYTES)
- return nan, inf, -inf
-
-NaN, PosInf, NegInf = _floatconstants()
-
-
-def linecol(doc, pos):
- lineno = doc.count('\n', 0, pos) + 1
- if lineno == 1:
- colno = pos
- else:
- colno = pos - doc.rindex('\n', 0, pos)
- return lineno, colno
-
-
-def errmsg(msg, doc, pos, end=None):
- # Note that this function is called from _speedups
- lineno, colno = linecol(doc, pos)
- if end is None:
- return '%s: line %d column %d (char %d)' % (msg, lineno, colno, pos)
- endlineno, endcolno = linecol(doc, end)
- return '%s: line %d column %d - line %d column %d (char %d - %d)' % (
- msg, lineno, colno, endlineno, endcolno, pos, end)
-
-
-_CONSTANTS = {
- '-Infinity': NegInf,
- 'Infinity': PosInf,
- 'NaN': NaN,
-}
-
-STRINGCHUNK = re.compile(r'(.*?)(["\\\x00-\x1f])', FLAGS)
-BACKSLASH = {
- '"': u'"', '\\': u'\\', '/': u'/',
- 'b': u'\b', 'f': u'\f', 'n': u'\n', 'r': u'\r', 't': u'\t',
-}
-
-DEFAULT_ENCODING = "utf-8"
-
-def py_scanstring(s, end, encoding=None, strict=True, _b=BACKSLASH, _m=STRINGCHUNK.match):
- """Scan the string s for a JSON string. End is the index of the
- character in s after the quote that started the JSON string.
- Unescapes all valid JSON string escape sequences and raises ValueError
- on attempt to decode an invalid string. If strict is False then literal
- control characters are allowed in the string.
-
- Returns a tuple of the decoded string and the index of the character in s
- after the end quote."""
- if encoding is None:
- encoding = DEFAULT_ENCODING
- chunks = []
- _append = chunks.append
- begin = end - 1
- while 1:
- chunk = _m(s, end)
- if chunk is None:
- raise ValueError(
- errmsg("Unterminated string starting at", s, begin))
- end = chunk.end()
- content, terminator = chunk.groups()
- # Content is contains zero or more unescaped string characters
- if content:
- if not isinstance(content, unicode):
- content = unicode(content, encoding)
- _append(content)
- # Terminator is the end of string, a literal control character,
- # or a backslash denoting that an escape sequence follows
- if terminator == '"':
- break
- elif terminator != '\\':
- if strict:
- msg = "Invalid control character %r at" % (terminator,)
- raise ValueError(msg, s, end)
- else:
- _append(terminator)
- continue
- try:
- esc = s[end]
- except IndexError:
- raise ValueError(
- errmsg("Unterminated string starting at", s, begin))
- # If not a unicode escape sequence, must be in the lookup table
- if esc != 'u':
- try:
- char = _b[esc]
- except KeyError:
- raise ValueError(
- errmsg("Invalid \\escape: %r" % (esc,), s, end))
- end += 1
- else:
- # Unicode escape sequence
- esc = s[end + 1:end + 5]
- next_end = end + 5
- if len(esc) != 4:
- msg = "Invalid \\uXXXX escape"
- raise ValueError(errmsg(msg, s, end))
- uni = int(esc, 16)
- # Check for surrogate pair on UCS-4 systems
- if 0xd800 <= uni <= 0xdbff and sys.maxunicode > 65535:
- msg = "Invalid \\uXXXX\\uXXXX surrogate pair"
- if not s[end + 5:end + 7] == '\\u':
- raise ValueError(errmsg(msg, s, end))
- esc2 = s[end + 7:end + 11]
- if len(esc2) != 4:
- raise ValueError(errmsg(msg, s, end))
- uni2 = int(esc2, 16)
- uni = 0x10000 + (((uni - 0xd800) << 10) | (uni2 - 0xdc00))
- next_end += 6
- char = unichr(uni)
- end = next_end
- # Append the unescaped character
- _append(char)
- return u''.join(chunks), end
-
-
-# Use speedup if available
-scanstring = c_scanstring or py_scanstring
-
-WHITESPACE = re.compile(r'[ \t\n\r]*', FLAGS)
-WHITESPACE_STR = ' \t\n\r'
-
-def JSONObject((s, end), encoding, strict, scan_once, object_hook, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
- pairs = {}
- # Use a slice to prevent IndexError from being raised, the following
- # check will raise a more specific ValueError if the string is empty
- nextchar = s[end:end + 1]
- # Normally we expect nextchar == '"'
- if nextchar != '"':
- if nextchar in _ws:
- end = _w(s, end).end()
- nextchar = s[end:end + 1]
- # Trivial empty object
- if nextchar == '}':
- return pairs, end + 1
- elif nextchar != '"':
- raise ValueError(errmsg("Expecting property name", s, end))
- end += 1
- while True:
- key, end = scanstring(s, end, encoding, strict)
-
- # To skip some function call overhead we optimize the fast paths where
- # the JSON key separator is ": " or just ":".
- if s[end:end + 1] != ':':
- end = _w(s, end).end()
- if s[end:end + 1] != ':':
- raise ValueError(errmsg("Expecting : delimiter", s, end))
-
- end += 1
-
- try:
- if s[end] in _ws:
- end += 1
- if s[end] in _ws:
- end = _w(s, end + 1).end()
- except IndexError:
- pass
-
- try:
- value, end = scan_once(s, end)
- except StopIteration:
- raise ValueError(errmsg("Expecting object", s, end))
- pairs[key] = value
-
- try:
- nextchar = s[end]
- if nextchar in _ws:
- end = _w(s, end + 1).end()
- nextchar = s[end]
- except IndexError:
- nextchar = ''
- end += 1
-
- if nextchar == '}':
- break
- elif nextchar != ',':
- raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
-
- try:
- nextchar = s[end]
- if nextchar in _ws:
- end += 1
- nextchar = s[end]
- if nextchar in _ws:
- end = _w(s, end + 1).end()
- nextchar = s[end]
- except IndexError:
- nextchar = ''
-
- end += 1
- if nextchar != '"':
- raise ValueError(errmsg("Expecting property name", s, end - 1))
-
- if object_hook is not None:
- pairs = object_hook(pairs)
- return pairs, end
-
-def JSONArray((s, end), scan_once, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
- values = []
- nextchar = s[end:end + 1]
- if nextchar in _ws:
- end = _w(s, end + 1).end()
- nextchar = s[end:end + 1]
- # Look-ahead for trivial empty array
- if nextchar == ']':
- return values, end + 1
- _append = values.append
- while True:
- try:
- value, end = scan_once(s, end)
- except StopIteration:
- raise ValueError(errmsg("Expecting object", s, end))
- _append(value)
- nextchar = s[end:end + 1]
- if nextchar in _ws:
- end = _w(s, end + 1).end()
- nextchar = s[end:end + 1]
- end += 1
- if nextchar == ']':
- break
- elif nextchar != ',':
- raise ValueError(errmsg("Expecting , delimiter", s, end))
-
- try:
- if s[end] in _ws:
- end += 1
- if s[end] in _ws:
- end = _w(s, end + 1).end()
- except IndexError:
- pass
-
- return values, end
-
-class JSONDecoder(object):
- """Simple JSON <http://json.org> decoder
-
- Performs the following translations in decoding by default:
-
- +---------------+-------------------+
- | JSON | Python |
- +===============+===================+
- | object | dict |
- +---------------+-------------------+
- | array | list |
- +---------------+-------------------+
- | string | unicode |
- +---------------+-------------------+
- | number (int) | int, long |
- +---------------+-------------------+
- | number (real) | float |
- +---------------+-------------------+
- | true | True |
- +---------------+-------------------+
- | false | False |
- +---------------+-------------------+
- | null | None |
- +---------------+-------------------+
-
- It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as
- their corresponding ``float`` values, which is outside the JSON spec.
-
- """
-
- def __init__(self, encoding=None, object_hook=None, parse_float=None,
- parse_int=None, parse_constant=None, strict=True):
- """``encoding`` determines the encoding used to interpret any ``str``
- objects decoded by this instance (utf-8 by default). It has no
- effect when decoding ``unicode`` objects.
-
- Note that currently only encodings that are a superset of ASCII work,
- strings of other encodings should be passed in as ``unicode``.
-
- ``object_hook``, if specified, will be called with the result
- of every JSON object decoded and its return value will be used in
- place of the given ``dict``. This can be used to provide custom
- deserializations (e.g. to support JSON-RPC class hinting).
-
- ``parse_float``, if specified, will be called with the string
- of every JSON float to be decoded. By default this is equivalent to
- float(num_str). This can be used to use another datatype or parser
- for JSON floats (e.g. decimal.Decimal).
-
- ``parse_int``, if specified, will be called with the string
- of every JSON int to be decoded. By default this is equivalent to
- int(num_str). This can be used to use another datatype or parser
- for JSON integers (e.g. float).
-
- ``parse_constant``, if specified, will be called with one of the
- following strings: -Infinity, Infinity, NaN.
- This can be used to raise an exception if invalid JSON numbers
- are encountered.
-
- """
- self.encoding = encoding
- self.object_hook = object_hook
- self.parse_float = parse_float or float
- self.parse_int = parse_int or int
- self.parse_constant = parse_constant or _CONSTANTS.__getitem__
- self.strict = strict
- self.parse_object = JSONObject
- self.parse_array = JSONArray
- self.parse_string = scanstring
- self.scan_once = make_scanner(self)
-
- def decode(self, s, _w=WHITESPACE.match):
- """Return the Python representation of ``s`` (a ``str`` or ``unicode``
- instance containing a JSON document)
-
- """
- obj, end = self.raw_decode(s, idx=_w(s, 0).end())
- end = _w(s, end).end()
- if end != len(s):
- raise ValueError(errmsg("Extra data", s, end, len(s)))
- return obj
-
- def raw_decode(self, s, idx=0):
- """Decode a JSON document from ``s`` (a ``str`` or ``unicode`` beginning
- with a JSON document) and return a 2-tuple of the Python
- representation and the index in ``s`` where the document ended.
-
- This can be used to decode a JSON document from a string that may
- have extraneous data at the end.
-
- """
- try:
- obj, end = self.scan_once(s, idx)
- except StopIteration:
- raise ValueError("No JSON object could be decoded")
- return obj, end
View
436 simplejson/encoder.py
@@ -1,436 +0,0 @@
-"""Implementation of JSONEncoder
-"""
-import re
-
-try:
- from simplejson._speedups import encode_basestring_ascii as c_encode_basestring_ascii
-except ImportError:
- c_encode_basestring_ascii = None
-try:
- from simplejson._speedups import make_encoder as c_make_encoder
-except ImportError:
- c_make_encoder = None
-
-ESCAPE = re.compile(r'[\x00-\x1f\\"\b\f\n\r\t]')
-ESCAPE_ASCII = re.compile(r'([\\"]|[^\ -~])')
-HAS_UTF8 = re.compile(r'[\x80-\xff]')
-ESCAPE_DCT = {
- '\\': '\\\\',
- '"': '\\"',
- '\b': '\\b',
- '\f': '\\f',
- '\n': '\\n',
- '\r': '\\r',
- '\t': '\\t',
-}
-for i in range(0x20):
- ESCAPE_DCT.setdefault(chr(i), '\\u%04x' % (i,))
-
-# Assume this produces an infinity on all machines (probably not guaranteed)
-INFINITY = float('1e66666')
-FLOAT_REPR = repr
-
-def encode_basestring(s):
- """Return a JSON representation of a Python string
-
- """
- def replace(match):
- return ESCAPE_DCT[match.group(0)]
- return '"' + ESCAPE.sub(replace, s) + '"'
-
-
-def py_encode_basestring_ascii(s):
- """Return an ASCII-only JSON representation of a Python string
-
- """
- if isinstance(s, str) and HAS_UTF8.search(s) is not None:
- s = s.decode('utf-8')
- def replace(match):
- s = match.group(0)
- try:
- return ESCAPE_DCT[s]
- except KeyError:
- n = ord(s)
- if n < 0x10000:
- return '\\u%04x' % (n,)
- else:
- # surrogate pair
- n -= 0x10000
- s1 = 0xd800 | ((n >> 10) & 0x3ff)
- s2 = 0xdc00 | (n & 0x3ff)
- return '\\u%04x\\u%04x' % (s1, s2)
- return '"' + str(ESCAPE_ASCII.sub(replace, s)) + '"'
-
-
-encode_basestring_ascii = c_encode_basestring_ascii or py_encode_basestring_ascii
-
-class JSONEncoder(object):
- """Extensible JSON <http://json.org> encoder for Python data structures.
-
- Supports the following objects and types by default:
-
- +-------------------+---------------+
- | Python | JSON |
- +===================+===============+
- | dict | object |
- +-------------------+---------------+
- | list, tuple | array |
- +-------------------+---------------+
- | str, unicode | string |
- +-------------------+---------------+
- | int, long, float | number |
- +-------------------+---------------+
- | True | true |
- +-------------------+---------------+
- | False | false |
- +-------------------+---------------+
- | None | null |
- +-------------------+---------------+
-
- To extend this to recognize other objects, subclass and implement a
- ``.default()`` method with another method that returns a serializable
- object for ``o`` if possible, otherwise it should call the superclass
- implementation (to raise ``TypeError``).
-
- """
- item_separator = ', '
- key_separator = ': '
- def __init__(self, skipkeys=False, ensure_ascii=True,
- check_circular=True, allow_nan=True, sort_keys=False,
- indent=None, separators=None, encoding='utf-8', default=None):
- """Constructor for JSONEncoder, with sensible defaults.
-
- If skipkeys is False, then it is a TypeError to attempt
- encoding of keys that are not str, int, long, float or None. If
- skipkeys is True, such items are simply skipped.
-
- If ensure_ascii is True, the output is guaranteed to be str
- objects with all incoming unicode characters escaped. If
- ensure_ascii is false, the output will be unicode object.
-
- If check_circular is True, then lists, dicts, and custom encoded
- objects will be checked for circular references during encoding to
- prevent an infinite recursion (which would cause an OverflowError).
- Otherwise, no such check takes place.
-
- If allow_nan is True, then NaN, Infinity, and -Infinity will be
- encoded as such. This behavior is not JSON specification compliant,
- but is consistent with most JavaScript based encoders and decoders.
- Otherwise, it will be a ValueError to encode such floats.
-
- If sort_keys is True, then the output of dictionaries will be
- sorted by key; this is useful for regression tests to ensure
- that JSON serializations can be compared on a day-to-day basis.
-
- If indent is a non-negative integer, then JSON array
- elements and object members will be pretty-printed with that
- indent level. An indent level of 0 will only insert newlines.
- None is the most compact representation.
-
- If specified, separators should be a (item_separator, key_separator)
- tuple. The default is (', ', ': '). To get the most compact JSON
- representation you should specify (',', ':') to eliminate whitespace.
-
- If specified, default is a function that gets called for objects
- that can't otherwise be serialized. It should return a JSON encodable
- version of the object or raise a ``TypeError``.
-
- If encoding is not None, then all input strings will be
- transformed into unicode using that encoding prior to JSON-encoding.
- The default is UTF-8.
-
- """
-
- self.skipkeys = skipkeys
- self.ensure_ascii = ensure_ascii
- self.check_circular = check_circular
- self.allow_nan = allow_nan
- self.sort_keys = sort_keys
- self.indent = indent
- if separators is not None:
- self.item_separator, self.key_separator = separators
- if default is not None:
- self.default = default
- self.encoding = encoding
-
- def default(self, o):
- """Implement this method in a subclass such that it returns
- a serializable object for ``o``, or calls the base implementation
- (to raise a ``TypeError``).
-
- For example, to support arbitrary iterators, you could
- implement default like this::
-
- def default(self, o):
- try:
- iterable = iter(o)
- except TypeError:
- pass
- else:
- return list(iterable)
- return JSONEncoder.default(self, o)
-
- """
- raise TypeError("%r is not JSON serializable" % (o,))
-
- def encode(self, o):
- """Return a JSON string representation of a Python data structure.
-
- >>> JSONEncoder().encode({"foo": ["bar", "baz"]})
- '{"foo": ["bar", "baz"]}'
-
- """
- # This is for extremely simple cases and benchmarks.
- if isinstance(o, basestring):
- if isinstance(o, str):
- _encoding = self.encoding
- if (_encoding is not None
- and not (_encoding == 'utf-8')):
- o = o.decode(_encoding)
- if self.ensure_ascii:
- return encode_basestring_ascii(o)
- else:
- return encode_basestring(o)
- # This doesn't pass the iterator directly to ''.join() because the
- # exceptions aren't as detailed. The list call should be roughly
- # equivalent to the PySequence_Fast that ''.join() would do.
- chunks = self.iterencode(o, _one_shot=True)
- if not isinstance(chunks, (list, tuple)):
- chunks = list(chunks)
- return ''.join(chunks)
-
- def iterencode(self, o, _one_shot=False):
- """Encode the given object and yield each string
- representation as available.
-
- For example::
-
- for chunk in JSONEncoder().iterencode(bigobject):
- mysocket.write(chunk)
-
- """
- if self.check_circular:
- markers = {}
- else:
- markers = None
- if self.ensure_ascii:
- _encoder = encode_basestring_ascii
- else:
- _encoder = encode_basestring
- if self.encoding != 'utf-8':
- def _encoder(o, _orig_encoder=_encoder, _encoding=self.encoding):
- if isinstance(o, str):
- o = o.decode(_encoding)
- return _orig_encoder(o)
-
- def floatstr(o, allow_nan=self.allow_nan, _repr=FLOAT_REPR, _inf=INFINITY, _neginf=-INFINITY):
- # Check for specials. Note that this type of test is processor- and/or
- # platform-specific, so do tests which don't depend on the internals.
-
- if o != o:
- text = 'NaN'
- elif o == _inf:
- text = 'Infinity'
- elif o == _neginf:
- text = '-Infinity'
- else:
- return _repr(o)
-
- if not allow_nan:
- raise ValueError("Out of range float values are not JSON compliant: %r"
- % (o,))
-
- return text
-
-
- if _one_shot and c_make_encoder is not None and not self.indent and not self.sort_keys:
- _iterencode = c_make_encoder(
- markers, self.default, _encoder, self.indent,
- self.key_separator, self.item_separator, self.sort_keys,
- self.skipkeys, self.allow_nan)
- else:
- _iterencode = _make_iterencode(
- markers, self.default, _encoder, self.indent, floatstr,
- self.key_separator, self.item_separator, self.sort_keys,
- self.skipkeys, _one_shot)
- return _iterencode(o, 0)
-
-def _make_iterencode(markers, _default, _encoder, _indent, _floatstr, _key_separator, _item_separator, _sort_keys, _skipkeys, _one_shot,
- ## HACK: hand-optimized bytecode; turn globals into locals
- False=False,
- True=True,
- ValueError=ValueError,
- basestring=basestring,
- dict=dict,
- float=float,
- id=id,
- int=int,
- isinstance=isinstance,
- list=list,
- long=long,
- str=str,
- tuple=tuple,
- ):
-
- def _iterencode_list(lst, _current_indent_level):
- if not lst:
- yield '[]'
- return
- if markers is not None:
- markerid = id(lst)
- if markerid in markers:
- raise ValueError("Circular reference detected")
- markers[markerid] = lst
- buf = '['
- if _indent is not None:
- _current_indent_level += 1
- newline_indent = '\n' + (' ' * (_indent * _current_indent_level))
- separator = _item_separator + newline_indent
- buf += newline_indent
- else:
- newline_indent = None
- separator = _item_separator
- first = True
- for value in lst:
- if first:
- first = False
- else:
- buf = separator
- if isinstance(value, basestring):
- yield buf + _encoder(value)
- elif value is None:
- yield buf + 'null'
- elif value is True:
- yield buf + 'true'
- elif value is False:
- yield buf + 'false'
- elif isinstance(value, (int, long)):
- yield buf + str(value)
- elif isinstance(value, float):
- yield buf + _floatstr(value)
- else:
- yield buf
- if isinstance(value, (list, tuple)):
- chunks = _iterencode_list(value, _current_indent_level)
- elif isinstance(value, dict):
- chunks = _iterencode_dict(value, _current_indent_level)
- else:
- chunks = _iterencode(value, _current_indent_level)
- for chunk in chunks:
- yield chunk
- if newline_indent is not None:
- _current_indent_level -= 1
- yield '\n' + (' ' * (_indent * _current_indent_level))
- yield ']'
- if markers is not None:
- del markers[markerid]
-
- def _iterencode_dict(dct, _current_indent_level):
- if not dct:
- yield '{}'
- return
- if markers is not None:
- markerid = id(dct)
- if markerid in markers:
- raise ValueError("Circular reference detected")
- markers[markerid] = dct
- yield '{'
- if _indent is not None:
- _current_indent_level += 1
- newline_indent = '\n' + (' ' * (_indent * _current_indent_level))
- item_separator = _item_separator + newline_indent
- yield newline_indent
- else:
- newline_indent = None
- item_separator = _item_separator
- first = True
- if _sort_keys:
- items = dct.items()
- items.sort(key=lambda kv: kv[0])
- else:
- items = dct.iteritems()
- for key, value in items:
- if isinstance(key, basestring):
- pass
- # JavaScript is weakly typed for these, so it makes sense to
- # also allow them. Many encoders seem to do something like this.
- elif isinstance(key, float):
- key = _floatstr(key)
- elif isinstance(key, (int, long)):
- key = str(key)
- elif key is True:
- key = 'true'
- elif key is False:
- key = 'false'
- elif key is None:
- key = 'null'
- elif _skipkeys:
- continue
- else:
- raise TypeError("key %r is not a string" % (key,))
- if first:
- first = False
- else:
- yield item_separator
- yield _encoder(key)
- yield _key_separator
- if isinstance(value, basestring):
- yield _encoder(value)
- elif value is None:
- yield 'null'
- elif value is True:
- yield 'true'
- elif value is False:
- yield 'false'
- elif isinstance(value, (int, long)):
- yield str(value)
- elif isinstance(value, float):
- yield _floatstr(value)
- else:
- if isinstance(value, (list, tuple)):
- chunks = _iterencode_list(value, _current_indent_level)
- elif isinstance(value, dict):
- chunks = _iterencode_dict(value, _current_indent_level)
- else:
- chunks = _iterencode(value, _current_indent_level)
- for chunk in chunks:
- yield chunk
- if newline_indent is not None:
- _current_indent_level -= 1
- yield '\n' + (' ' * (_indent * _current_indent_level))
- yield '}'
- if markers is not None:
- del markers[markerid]
-
- def _iterencode(o, _current_indent_level):
- if isinstance(o, basestring):
- yield _encoder(o)
- elif o is None:
- yield 'null'
- elif o is True:
- yield 'true'
- elif o is False:
- yield 'false'
- elif isinstance(o, (int, long)):
- yield str(o)
- elif isinstance(o, float):
- yield _floatstr(o)
- elif isinstance(o, (list, tuple)):
- for chunk in _iterencode_list(o, _current_indent_level):
- yield chunk
- elif isinstance(o, dict):
- for chunk in _iterencode_dict(o, _current_indent_level):
- yield chunk
- else:
- if markers is not None:
- markerid = id(o)
- if markerid in markers:
- raise ValueError("Circular reference detected")
- markers[markerid] = o
- o = _default(o)
- for chunk in _iterencode(o, _current_indent_level):
- yield chunk
- if markers is not None:
- del markers[markerid]
-
- return _iterencode
View
65 simplejson/scanner.py
@@ -1,65 +0,0 @@
-"""JSON token scanner
-"""
-import re
-try:
- from simplejson._speedups import make_scanner as c_make_scanner
-except ImportError:
- c_make_scanner = None
-
-__all__ = ['make_scanner']
-
-NUMBER_RE = re.compile(
- r'(-?(?:0|[1-9]\d*))(\.\d+)?([eE][-+]?\d+)?',
- (re.VERBOSE | re.MULTILINE | re.DOTALL))
-
-def py_make_scanner(context):
- parse_object = context.parse_object
- parse_array = context.parse_array
- parse_string = context.parse_string
- match_number = NUMBER_RE.match
- encoding = context.encoding
- strict = context.strict
- parse_float = context.parse_float
- parse_int = context.parse_int
- parse_constant = context.parse_constant
- object_hook = context.object_hook
-
- def _scan_once(string, idx):
- try:
- nextchar = string[idx]
- except IndexError:
- raise StopIteration
-
- if nextchar == '"':
- return parse_string(string, idx + 1, encoding, strict)
- elif nextchar == '{':
- return parse_object((string, idx + 1), encoding, strict, _scan_once, object_hook)
- elif nextchar == '[':
- return parse_array((string, idx + 1), _scan_once)
- elif nextchar == 'n' and string[idx:idx + 4] == 'null':
- return None, idx + 4
- elif nextchar == 't' and string[idx:idx + 4] == 'true':
- return True, idx + 4
- elif nextchar == 'f' and string[idx:idx + 5] == 'false':
- return False, idx + 5
-
- m = match_number(string, idx)
- if m is not None:
- integer, frac, exp = m.groups()
- if frac or exp:
- res = parse_float(integer + (frac or '') + (exp or ''))
- else:
- res = parse_int(integer)
- return res, m.end()
- elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':
- return parse_constant('NaN'), idx + 3
- elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':
- return parse_constant('Infinity'), idx + 8
- elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':
- return parse_constant('-Infinity'), idx + 9
- else:
- raise StopIteration
-
- return _scan_once
-
-make_scanner = c_make_scanner or py_make_scanner
View
35 simplejson/tool.py
@@ -1,35 +0,0 @@
-r"""Using simplejson from the shell to validate and
-pretty-print::
-
- $ echo '{"json":"obj"}' | python -msimplejson.tool
- {
- "json": "obj"
- }
- $ echo '{ 1.2:3.4}' | python -msimplejson.tool
- Expecting property name: line 1 column 2 (char 2)
-"""
-import simplejson
-
-def main():
- import sys
- if len(sys.argv) == 1:
- infile = sys.stdin
- outfile = sys.stdout
- elif len(sys.argv) == 2:
- infile = open(sys.argv[1], 'rb')
- outfile = sys.stdout
- elif len(sys.argv) == 3:
- infile = open(sys.argv[1], 'rb')
- outfile = open(sys.argv[2], 'wb')
- else:
- raise SystemExit("%s [infile [outfile]]" % (sys.argv[0],))
- try:
- obj = simplejson.load(infile)
- except ValueError, e:
- raise SystemExit(e)
- simplejson.dump(obj, outfile, sort_keys=True, indent=4)
- outfile.write('\n')
-
-
-if __name__ == '__main__':
- main()

0 comments on commit 1a28a85

Please sign in to comment.
Something went wrong with that request. Please try again.