Skip to content

Commit

Permalink
ENH: Add enforcement for whitespace cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
aragilar committed Jun 11, 2018
1 parent bfaa83a commit 5397716
Show file tree
Hide file tree
Showing 72 changed files with 298 additions and 433 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ MANIFEST
/*.egg
.eggs/
*.so
*~
*~
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v1.2.3
hooks:
- id: end-of-file-fixer
- id: trailing-whitespace
1 change: 1 addition & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ env:
- TOXENV=docs
- TOXENV=check-manifest
- TOXENV=checkreadme
- TOXENV=pre-commit

matrix:
include:
Expand Down
4 changes: 2 additions & 2 deletions ANN.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The h5py team is happy to announce the availability of h5py 2.5.0.
This release introduces experimental support for the highly-anticipated
"Single Writer Multiple Reader" (SWMR) feature in the upcoming HDF5 1.10
release. SWMR allows sharing of a single HDF5 file between multiple processes
without the complexity of MPI or multiprocessing-based solutions.
without the complexity of MPI or multiprocessing-based solutions.

This is an experimental feature that should NOT be used in production code.
We are interested in getting feedback from the broader community with respect
Expand Down Expand Up @@ -42,7 +42,7 @@ Changes
* Fixed an issue with 64-bit float VLENS
* Cython warning cleanups related to "const"
* Entire code base ported to "six"; 2to3 removed from setup.py


Acknowledgements
----------------
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,4 @@ recursive-exclude other *
recursive-exclude .github *
exclude pavement.py
exclude *.yml
exclude *.yaml
2 changes: 1 addition & 1 deletion ci/appveyor/vs2008_patch/setup_x64.bat
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ regedit /s x64\656d8763-2429-11d7-8bf6-00b0d03daa06.reg
regedit /s x64\656d8766-2429-11d7-8bf6-00b0d03daa06.reg

copy "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcpackages\AMD64.VCPlatform.config" "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcpackages\AMD64.VCPlatform.Express.config"
copy "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.config" "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.Express.config"
copy "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.config" "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.Express.config"
2 changes: 1 addition & 1 deletion ci/appveyor/vs2008_patch/setup_x86.bat
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ regedit /s x86\656d8763-2429-11d7-8bf6-00b0d03daa06.reg
regedit /s x86\656d8766-2429-11d7-8bf6-00b0d03daa06.reg

copy "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcpackages\AMD64.VCPlatform.config" "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcpackages\AMD64.VCPlatform.Express.config"
copy "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.config" "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.Express.config"
copy "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.config" "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcpackages\Itanium.VCPlatform.Express.config"
17 changes: 7 additions & 10 deletions docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Variable length array Any supported type
Unsupported types:

========================= ============================================
Type Status
Type Status
========================= ============================================
HDF5 "time" type
NumPy "U" strings No HDF5 equivalent
Expand Down Expand Up @@ -61,7 +61,7 @@ What file drivers are available?
--------------------------------

A number of different HDF5 "drivers", which provide different modes of access
to the filesystem, are accessible in h5py via the high-level interface. The
to the filesystem, are accessible in h5py via the high-level interface. The
currently supported drivers are:

=================================== =========================================== ============================
Expand Down Expand Up @@ -95,10 +95,10 @@ This layer is object-oriented with respect to HDF5 identifiers, supports
reference counting, automatic translation between NumPy and HDF5 type objects,
translation between the HDF5 error stack and Python exceptions, and more.

This greatly simplifies the design of the complicated high-level interface, by
This greatly simplifies the design of the complicated high-level interface, by
relying on the "Pythonicity" of the C API wrapping.

There's also a PyTables perspective on this question at the
There's also a PyTables perspective on this question at the
`PyTables FAQ <http://www.pytables.org/FAQ.html#how-does-pytables-compare-with-the-h5py-project>`_.


Expand Down Expand Up @@ -138,14 +138,14 @@ Appending data to a dataset
The short response is that h5py is NumPy-like, not database-like. Unlike the
HDF5 packet-table interface (and PyTables), there is no concept of appending
rows. Rather, you can expand the shape of the dataset to fit your needs. For
example, if I have a series of time traces 1024 points long, I can create an
example, if I have a series of time traces 1024 points long, I can create an
extendable dataset to store them:

>>> dset = myfile.create_dataset("MyDataset", (10, 1024), maxshape=(None, 1024))
>>> dset.shape
(10,1024)

The keyword argument "maxshape" tells HDF5 that the first dimension of the
The keyword argument "maxshape" tells HDF5 that the first dimension of the
dataset can be expanded to any size, while the second dimension is limited to a
maximum size of 1024. We create the dataset with room for an initial ensemble
of 10 time traces. If we later want to store 10 more time traces, the dataset
Expand All @@ -163,7 +163,7 @@ Each axis can be resized up to the maximum values in "maxshape". Things to note:

Unicode
-------
As of h5py 2.0.0, Unicode is supported for file names as well as for objects
As of h5py 2.0.0, Unicode is supported for file names as well as for objects
in the file. When object names are read, they are returned as Unicode by default.

However, HDF5 has no predefined datatype to represent fixed-width UTF-16 or
Expand Down Expand Up @@ -205,6 +205,3 @@ containing a compiled, shared-library build of HDF5 (containing things like "inc
$ python setup.py test

Report any failing tests to the mailing list (h5py at googlegroups), or by filing a bug report at GitHub.



3 changes: 0 additions & 3 deletions docs/high/attr.rst
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,3 @@ Reference
:type name: String

:param value: New value. Will be put through ``numpy.array(value)``.



2 changes: 1 addition & 1 deletion docs/high/file.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ of supported drivers and their options:
'sec2'
Unbuffered, optimized I/O using standard POSIX functions.

'stdio'
'stdio'
Buffered I/O using functions from stdio.h.

'core'
Expand Down
4 changes: 2 additions & 2 deletions docs/high/group.rst
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ is to create an :ref:`HDF5 datasets <dataset>`::
>>> out = grp["name"]
>>> out
<HDF5 dataset "name": shape (), type "<i8">

When the object being stored is an existing Group or Dataset, a new link is
made to the object::

Expand Down Expand Up @@ -430,7 +430,7 @@ Link classes

:param filename: Name of the file to which the link points
:type filename: String

:param path: Path to the object in the external file.
:type path: String

Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Advanced topics

.. toctree::
:maxdepth: 1

config
special
strings
Expand Down
7 changes: 3 additions & 4 deletions docs/licenses.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ Copyright Notice and Statement for the h5py Project
documentation and/or other materials provided with the
distribution.

c. Neither the name of the author nor the names of contributors may
be used to endorse or promote products derived from this software
c. Neither the name of the author nor the names of contributors may
be used to endorse or promote products derived from this software
without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
Expand Down Expand Up @@ -179,7 +179,7 @@ stdint.h (Windows version) License
EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Expand Down Expand Up @@ -229,4 +229,3 @@ Python license

#. By copying, installing or otherwise using Python Python 2.7.5, Licensee agrees
to be bound by the terms and conditions of this License Agreement.

1 change: 0 additions & 1 deletion docs/mpi.rst
Original file line number Diff line number Diff line change
Expand Up @@ -147,4 +147,3 @@ More information
Parallel HDF5 is a new feature in h5py. If you have any questions, feel free to
ask on the mailing list (h5py at google groups). We welcome bug reports,
enhancements and general inquiries.

2 changes: 0 additions & 2 deletions docs/quick.rst
Original file line number Diff line number Diff line change
Expand Up @@ -185,5 +185,3 @@ implements the dictionary interface::
True

For more, see :ref:`attributes`.


4 changes: 1 addition & 3 deletions docs/refs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ opening any other object:
Using region references
-----------------------

Region references always contain a selection. You create them using the
Region references always contain a selection. You create them using the
dataset property "regionref" and standard NumPy slicing syntax:

>>> myds = myfile.create_dataset('dset', (200,200))
Expand Down Expand Up @@ -119,5 +119,3 @@ indicate whether or not it is null:
>>> nullref = ref_dataset[50]
>>> print bool(nullref)
False


4 changes: 2 additions & 2 deletions docs/special.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ dtype. In h5py, variable-length strings are mapped to object arrays. A
small amount of metadata attached to an "O" dtype tells h5py that its contents
should be converted to VL strings when stored in the file.

Existing VL strings can be read and written to with no additional effort;
Existing VL strings can be read and written to with no additional effort;
Python strings and fixed-length NumPy strings can be auto-converted to VL
data and stored.

Expand Down Expand Up @@ -100,7 +100,7 @@ arrays::

>>> dset[0:2]
array([array([1, 2, 3], dtype=int32), array([1, 2, 3, 4, 5], dtype=int32)], dtype=object)


Enumerated types
----------------
Expand Down
26 changes: 12 additions & 14 deletions docs/swmr.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,25 @@ Starting with version 2.5.0, h5py includes support for the HDF5 SWMR features.
What is SWMR?
-------------

The SWMR features allow simple concurrent reading of a HDF5 file while it is
The SWMR features allow simple concurrent reading of a HDF5 file while it is
being written from another process. Prior to this feature addition it was not
possible to do this as the file data and meta-data would not be synchronised
and attempts to read a file which was open for writing would fail or result in
garbage data.

A file which is being written to in SWMR mode is guaranteed to always be in a
valid (non-corrupt) state for reading. This has the added benefit of leaving a
file in a valid state even if the writing application crashes before closing
valid (non-corrupt) state for reading. This has the added benefit of leaving a
file in a valid state even if the writing application crashes before closing
the file properly.

This feature has been implemented to work with independent writer and reader
processes. No synchronisation is required between processes and it is up to the
user to implement either a file polling mechanism, inotify or any other IPC
user to implement either a file polling mechanism, inotify or any other IPC
mechanism to notify when data has been written.

The SWMR functionality requires use of the latest HDF5 file format: v110. In
practice this implies using at least HDF5 1.10 (this can be checked via
`h5py.info`) and setting the libver bounding to "latest" when opening or
practice this implies using at least HDF5 1.10 (this can be checked via
`h5py.info`) and setting the libver bounding to "latest" when opening or
creating the file.


Expand Down Expand Up @@ -87,30 +87,30 @@ Examples
--------

In addition to the above example snippets, a few more complete examples can be
found in the examples folder. These examples are described in the following
found in the examples folder. These examples are described in the following
sections

Dataset monitor with inotify
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The inotify example demonstrate how to use SWMR in a reading application which
monitors live progress as a dataset is being written by another process. This
example uses the the linux inotify
(`pyinotify <https://pypi.python.org/pypi/pyinotify>`_ python bindings) to
example uses the the linux inotify
(`pyinotify <https://pypi.python.org/pypi/pyinotify>`_ python bindings) to
receive a signal each time the target file has been updated.

.. literalinclude:: ../examples/swmr_inotify_example.py

Multiprocess concurrent write and read
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The SWMR multiprocess example starts two concurrent child processes:
The SWMR multiprocess example starts two concurrent child processes:
a writer and a reader.
The writer process first creates the target file and dataset. Then it switches
the file into SWMR mode and the reader process is notified (with a
multiprocessing.Event) that it is safe to open the file for reading.

The writer process then continue to append chunks to the dataset. After each
The writer process then continue to append chunks to the dataset. After each
write it notifies the reader that new data has been written. Whether the new
data is visible in the file at this point is subject to OS and file system
latencies.
Expand All @@ -126,7 +126,7 @@ it will drop out of the loop and exit.
The example output below (from a virtual Ubuntu machine) illustrate some
latency between the writer and reader::

python examples/swmr_multiprocess.py
python examples/swmr_multiprocess.py
INFO 2015-02-26 18:05:03,195 root Starting reader
INFO 2015-02-26 18:05:03,196 root Starting reader
INFO 2015-02-26 18:05:03,197 reader Waiting for initial event
Expand All @@ -150,5 +150,3 @@ latency between the writer and reader::
INFO 2015-02-26 18:05:03,217 reader Read dset shape: (20,)
INFO 2015-02-26 18:05:03,218 reader Read dset shape: (20,)
INFO 2015-02-26 18:05:03,219 root Waiting for reader to finish


9 changes: 0 additions & 9 deletions docs/whatsnew/2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,12 +166,3 @@ Known issues
* There are reports of crashes related to storing object and region references.
If this happens to you, please post on the mailing list or contact the h5py
author directly.









3 changes: 1 addition & 2 deletions docs/whatsnew/2.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Support for Parallel HDF5
-------------------------

On UNIX platforms, you can now take advantage of MPI and Parallel HDF5.
Cython, ``mpi4py`` and an MPI-enabled build of HDF5 are required..
Cython, ``mpi4py`` and an MPI-enabled build of HDF5 are required..
See :ref:`parallel` in the documentation for details.


Expand Down Expand Up @@ -98,4 +98,3 @@ The method ``Group.move`` allows this to be performed in one step::
>>> f.move('a', 'b')

Both the source and destination must be in the same file.

6 changes: 3 additions & 3 deletions docs/whatsnew/2.4.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ Build system changes

The setup.py-based build system has been reworked to be more maintainable, and
to fix certain long-standing bugs. As a consequence, the options to setup.py
have changed; a new top-level "configure" command handles options like
have changed; a new top-level "configure" command handles options like
``--hdf5=/path/to/hdf5`` and ``--mpi``. Setup.py now works correctly under
Python 3 when these options are used.

Cython (0.17+) is now required when building from source on all platforms;
the .c files are no longer shipped in the UNIX release. The minimum NumPy
the .c files are no longer shipped in the UNIX release. The minimum NumPy
version is now 1.6.1.

Files will now auto-close
-------------------------

Files are now automatically closed when all objects within them
are unreachable. Previously, if File.close() was not explicitly called,
files would remain open and "leaks" were possible if the File object
Expand Down
1 change: 0 additions & 1 deletion docs/whatsnew/2.5.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,3 @@ Other changes
Acknowledgements
----------------

1 change: 0 additions & 1 deletion docs/whatsnew/2.7.rst
Original file line number Diff line number Diff line change
Expand Up @@ -92,4 +92,3 @@ Other changes
Acknowledgements
----------------

3 changes: 0 additions & 3 deletions docs_api/h5.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,3 @@ Module constants
.. data:: ITER_DEC

Traverse index in decreasing order



2 changes: 0 additions & 2 deletions docs_api/h5a.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,3 @@ Attribute objects

.. autoclass:: AttrID
:members:


Loading

0 comments on commit 5397716

Please sign in to comment.