Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: dstuebe/pyon
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: ooici-dm/pyon
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Able to merge. These branches can be automatically merged.
12 contributors
Loading
Showing with 10,387 additions and 2,493 deletions.
  1. +2 −0 .gitignore
  2. +1 −1 README
  3. +1 −1 examples/hello_client.py
  4. +51 −0 examples/service/arg_check_example.py
  5. +4 −4 examples/stream/stream_binder.py
  6. +1 −1 examples/stream/stream_producer.py
  7. +1 −1 extern/ion-definitions
  8. +459 −0 prototype/hdf/hdf_array_iterator.py
  9. +147 −96 prototype/hdf/hdf_codec.py
  10. +693 −0 prototype/hdf/test/test_hdf_array_iterator.py
  11. +21 −7 prototype/hdf/test/test_science_object_codec.py
  12. +614 −124 prototype/sci_data/constructor_apis.py
  13. +0 −398 prototype/sci_data/ctd_stream.py
  14. +556 −0 prototype/sci_data/stream_defs.py
  15. +73 −0 prototype/sci_data/stream_parser.py
  16. +7 −0 prototype/transforms/__init__.py
  17. +38 −0 prototype/transforms/dct.py
  18. +72 −0 prototype/transforms/linear.py
  19. +61 −19 pyon/agent/agent.py
  20. +0 −11 pyon/container/apps.py
  21. +133 −37 pyon/container/cc.py
  22. +86 −21 pyon/container/procs.py
  23. +3 −1 pyon/container/shell_api.py
  24. +1 −1 pyon/container/test/test_apps.py
  25. +32 −4 pyon/container/test/test_procs.py
  26. +10 −2 pyon/core/bootstrap.py
  27. +50 −2 pyon/core/exception.py
  28. +98 −7 pyon/core/governance/governance_controller.py
  29. +6 −3 pyon/core/governance/governance_dispatcher.py
  30. +10 −9 pyon/core/governance/governance_interceptor.py
  31. +253 −0 pyon/core/governance/negotiate_request.py
  32. +15 −0 pyon/core/governance/policy/empty_policy_set.xml
  33. +94 −28 pyon/core/governance/policy/policy_decision.py
  34. +17 −14 pyon/core/governance/policy/policy_interceptor.py
  35. +307 −48 pyon/core/governance/policy/sample_policies.xml
  36. 0 pyon/core/governance/policy/xacml/__init__.py
  37. +101 −0 pyon/core/governance/policy/xacml/and_function.py
  38. +69 −0 pyon/core/governance/policy/xacml/not_function.py
  39. +9 −3 pyon/core/governance/test/test_governance.py
  40. +12 −1 pyon/core/interceptor/codec.py
  41. +43 −2 pyon/core/interceptor/encode.py
  42. +12 −0 pyon/core/interceptor/interceptor.py
  43. +27 −0 pyon/core/interceptor/signature.py
  44. +8 −8 pyon/core/interceptor/test/interceptor_test.py
  45. +60 −30 pyon/core/object.py
  46. +34 −2 pyon/core/registry.py
  47. +3 −8 pyon/core/security/authentication.py
  48. +62 −28 pyon/datastore/couchdb/couchdb_config.py
  49. +155 −115 pyon/datastore/couchdb/couchdb_datastore.py
  50. +22 −11 pyon/datastore/datastore.py
  51. +16 −12 pyon/datastore/mockdb/mockdb_datastore.py
  52. +16 −14 pyon/datastore/test/test_datastores.py
  53. +152 −434 pyon/event/event.py
  54. +124 −211 pyon/event/test/test_event.py
  55. +117 −32 pyon/ion/directory.py
  56. +6 −4 pyon/ion/endpoint.py
  57. +494 −73 pyon/ion/exchange.py
  58. +166 −97 pyon/ion/resource.py
  59. +318 −0 pyon/ion/resregistry.py
  60. +5 −1 pyon/ion/streamproc.py
  61. +34 −2 pyon/ion/test/test_directory.py
  62. +414 −0 pyon/ion/test/test_exchange.py
  63. +31 −23 pyon/ion/test/test_resource.py
  64. +71 −0 pyon/ion/test/test_resregistry.py
  65. +126 −2 pyon/ion/transform.py
  66. +239 −90 pyon/net/channel.py
  67. +339 −173 pyon/net/endpoint.py
  68. +60 −9 pyon/net/messaging.py
  69. +344 −136 pyon/net/test/test_channel.py
  70. +558 −35 pyon/net/test/test_endpoint.py
  71. +86 −8 pyon/net/test/test_messaging.py
  72. +13 −5 pyon/net/test/test_speed.py
  73. +179 −0 pyon/net/transport.py
  74. +4 −5 pyon/public.py
  75. +10 −2 pyon/service/service.py
  76. +186 −0 pyon/util/arg_check.py
  77. +1 −1 pyon/util/async.py
  78. +3 −1 pyon/util/config.py
  79. +45 −5 pyon/util/containers.py
  80. +23 −0 pyon/util/file_sys.py
  81. +594 −0 pyon/util/insulate.py
  82. +14 −20 pyon/util/int_test.py
  83. +35 −0 pyon/util/log.py
  84. +161 −0 pyon/util/lru_cache.py
  85. +51 −0 pyon/util/nose_gevent_profiler.py
  86. +27 −0 pyon/util/profiler.py
  87. +134 −0 pyon/util/pycc_plugin.py
  88. +222 −0 pyon/util/sflow.py
  89. +81 −0 pyon/util/test/test_arg_check.py
  90. +33 −0 pyon/util/test/test_profiler.py
  91. +13 −4 scripts/control_cc.py
  92. +239 −32 scripts/generate_interfaces.py
  93. +57 −10 scripts/pycc.py
  94. +12 −4 setup.py
2 changes: 2 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -34,3 +34,5 @@ unittest.out
interface/*
cc-pid*
.coverage

cover/
2 changes: 1 addition & 1 deletion README
View file
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@ Install the following if not yet present:

* OS Packages and package management:
For Mac, use homebrew
> /usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
> /usr/bin/ruby -e "$(/usr/bin/curl -fksSL https://raw.github.com/mxcl/homebrew/master/Library/Contributions/install_homebrew.rb)"
- python 2.7.2
Download and use installer on a Mac.
If you need to build from source (like on some linux distributions), enable the following in the build:
2 changes: 1 addition & 1 deletion examples/hello_client.py
View file
Original file line number Diff line number Diff line change
@@ -31,5 +31,5 @@ def hello_client(container, actor_id='anonymous', org_id='no-ooi', text='mytext

container = Container()
container.start() # :(
hello_client(container, user_id='shenrie',org_id='ooi')
hello_client(container, actor_id='shenrie',org_id='ooi')
container.stop()
51 changes: 51 additions & 0 deletions examples/service/arg_check_example.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#!/usr/bin/env python
'''
@author Luke Campbel <LCampbell@ASAScience.com>
@file
@date 03/27/12 15:30
@description DESCRIPTION
'''
from pyon.util.arg_check import validateIsInstance, validateIn, validateEqual, validateTrue


class ArgCheckService(object):
'''
Example Service illustrating how to use the various validateion mechanisms
'''
def __init__(self):
pass

def pass_integer(self, val=''):
'''
Say you were expecting an integer from the client...
'''
validateIsInstance(val,int,'Value is not an integer.')
return val

def pass_float(self, val=1.0):
'''
Say you were expecting a float from the client
'''
validateIsInstance(val,float,'Value is not a float.')
return val

def handle_list(self, needle, haystack):
'''
You needed to be certain that something was in the list or dict
'''
validateIn(needle,haystack,'Can\'t find %s in %s.' % (needle, haystack))
return needle

def check_equality(self, a,b):
'''
You needed to be sure that two items we're equivalent
'''
validateEqual(a,b,'%s != %s' %(str(a), str(b)))
return True

def list_len(self,l):
'''
You needed to be certain that a list had len >0
'''
validateTrue(len(l)>0, 'list=%s was empty.' % str(l))

8 changes: 4 additions & 4 deletions examples/stream/stream_binder.py
View file
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
from pyon.net.channel import SubscriberChannel

from pyon.core import bootstrap

from pyon.net.transport import NameTrio

class BindingChannel(SubscriberChannel):
"""
@@ -36,7 +36,7 @@ class BindingChannel(SubscriberChannel):
"""
def _declare_queue(self, queue):

self._recv_name = (self._recv_name[0], '.'.join(self._recv_name))
self._recv_name = NameTrio(self._recv_name.exchange, '.'.join((self._recv_name.exchange, self._recv_name.queue)))


class StreamBinder(SimpleProcess):
@@ -50,9 +50,9 @@ def on_start(self):
binding = self.CFG.get('args',{}).get('binding',None)

# Create scoped exchange name
XP = '.'.join([bootstrap.sys_name,'science_data'])
XP = '.'.join([bootstrap.get_sys_name(),'science_data'])

self.channel = self.container.node.channel(BindingChannel)
self.channel.setup_listener((XP,queue_name),binding=binding)
self.channel.setup_listener(NameTrio(XP,queue_name),binding=binding)

# How do we make this process end now?
2 changes: 1 addition & 1 deletion examples/stream/stream_producer.py
View file
Original file line number Diff line number Diff line change
@@ -48,7 +48,7 @@ def _trigger_func(self):
routing_key = self.CFG.get('stream_producer').get('routing_key')

# Create scoped exchange name
XP = '.'.join([bootstrap.sys_name,'science_data'])
XP = '.'.join([bootstrap.get_sys_name(),'science_data'])

pub = ProcessPublisher(node=self.container.node, name=(XP,routing_key), process=self)
num = 1
2 changes: 1 addition & 1 deletion extern/ion-definitions
Submodule ion-definitions updated 98 files
+2 −0 objects/data/coi/directory.yml
+37 −0 objects/data/coi/org.yml
+4 −0 objects/data/coi/policy.yml
+1 −0 objects/data/dm/ingestion.yml
+9 −5 objects/data/dm/science_data_model.yml
+1 −0 objects/data/dm/subscription.yml
+40 −9 objects/data/eoi/externaldata.yml
+103 −19 objects/data/event.yml
+13 −0 objects/data/last_update.yml
+77 −17 objects/data/resource.yml
+1 −4 objects/data/sa/dataprocess.yml
+6 −5 objects/data/sa/dataproduct.yml
+1 −0 objects/data/sa/device.yml
+17 −6 objects/data/sa/instrument.yml
+18 −65 objects/data/sa/marinefacility.yml
+3 −3 objects/data/sa/platform.yml
+2 −2 objects/data/sa/sensor.yml
+72 −0 objects/data/ux/ui.yml
+76 −0 objects/services/ans/visualization_service.yml
+4 −3 objects/services/bootstrap_service.yml
+13 −8 objects/services/cei/process_dispatcher_service.yml
+4 −1 objects/services/coi/datastore_service.yml
+10 −0 objects/services/coi/identity_management_service.yml
+217 −22 objects/services/coi/org_management_service.yml
+101 −30 objects/services/coi/policy_management_service.yml
+118 −8 objects/services/coi/resource_registry_service.yml
+26 −3 objects/services/container_agent.yml
+24 −3 objects/services/dm/pubsub_management_service.yml
+41 −10 objects/services/dm/user_notification_service.yml
+25 −3 objects/services/resource_agent.yml
+171 −56 objects/services/sa/data_acquisition_management_service.yml
+46 −3 objects/services/sa/data_process_management_service.yml
+12 −2 objects/services/sa/data_product_management_service.yml
+376 −70 objects/services/sa/instrument_management_service.yml
+235 −67 objects/services/sa/marine_facility_management_service.yml
+203 −0 objects/services/sa/observatory_management_service.yml
+84 −40 res/config/associations.yml
+14 −0 res/config/logging.pycc.yml
+14 −0 res/config/logging.yml
+20 −8 res/config/pyon.yml
+23 −0 res/config/resource_lifecycle.yml
+13 −0 res/deploy/examples/dispatcher.yml
+10 −3 res/deploy/examples/ingestion.yml
+12 −0 res/deploy/examples/ingestion_cache.yml
+23 −0 res/deploy/examples/viz_data_launcher.yml
+210 −0 res/deploy/r2as.yml
+16 −39 res/deploy/r2coi.yml
+65 −17 res/deploy/r2deploy.yml
+194 −0 res/deploy/r2deploy_no_bootstrap.yml
+33 −0 res/deploy/r2dm.yml
+17 −5 res/deploy/r2eoi.yml
+47 −6 res/deploy/r2gov.yml
+250 −0 res/deploy/r2lca.yml
+208 −0 res/deploy/r2lca_no_bootstrap.yml
+60 −5 res/deploy/r2sa.yml
+37 −0 res/deploy/viz.yml
+4 −0 res/preload/lca_demo/Attachment.csv
+7 −0 res/preload/lca_demo/DataProcess.csv
+9 −0 res/preload/lca_demo/DataProcessDefinition.csv
+16 −0 res/preload/lca_demo/DataProduct.csv
+5 −0 res/preload/lca_demo/DataProductLink.csv
+3 −0 res/preload/lca_demo/IngestionConfiguration.csv
+2 −0 res/preload/lca_demo/InstrumentAgent.csv
+3 −0 res/preload/lca_demo/InstrumentAgentInstance.csv
+3 −0 res/preload/lca_demo/InstrumentAgentInstance_localhost.csv
+4 −0 res/preload/lca_demo/InstrumentDevice.csv
+2 −0 res/preload/lca_demo/InstrumentModel.csv
+3 −0 res/preload/lca_demo/LogicalInstrument.csv
+4 −0 res/preload/lca_demo/LogicalPlatform.csv
+5 −0 res/preload/lca_demo/MarineFacility.csv
+4 −0 res/preload/lca_demo/PlatformDevice.csv
+7 −0 res/preload/lca_demo/PlatformModel.csv
+5 −0 res/preload/lca_demo/Site.csv
+13 −0 res/preload/lca_demo/StreamDefinition.csv
+14 −0 res/preload/lca_demo/User.csv
+15 −0 res/preload/lca_demo/UserRole.csv
+ − res/preload/lca_demo/attachments/att1.pdf
+ − res/preload/lca_demo/attachments/att2.png
+11 −0 res/preload/lca_demo/attachments/att3.csv
+149 −0 res/preload/lca_demo/ooi_assets/Report1_InstrumentLocations.csv
+59 −0 res/preload/lca_demo/ooi_assets/Report2_InstrumentTypes.csv
+636 −0 res/preload/lca_demo/ooi_assets/Report3_InstrumentTypeByLocation.csv
+67 −0 res/preload/lca_demo/ooi_assets/Report4_InstrumentsPerSite.csv
+772 −0 res/preload/lca_demo/ooi_assets/Report5_ReferenceDesignatorList.csv
+772 −0 res/preload/lca_demo/ooi_assets/Report6_ReferenceDesignatorListWithDepth.csv
+90 −0 res/preload/lca_demo/ui_assets/Architectural Source.csv
+598 −0 res/preload/lca_demo/ui_assets/Attribute.csv
+39 −0 res/preload/lca_demo/ui_assets/Block.csv
+ − res/preload/lca_demo/ui_assets/CIUX database schema.tiff
+14 −0 res/preload/lca_demo/ui_assets/Group.csv
+8 −0 res/preload/lca_demo/ui_assets/Representation block.csv
+4 −0 res/preload/lca_demo/ui_assets/Resource.csv
+298 −0 res/preload/lca_demo/ui_assets/Screen Label free.csv
+5 −0 res/preload/lca_demo/ui_assets/View.csv
+212 −0 res/preload/lca_demo/ui_assets/_jt_Block_Attribute.csv
+46 −0 res/preload/lca_demo/ui_assets/_jt_Block_Representation.csv
+39 −0 res/preload/lca_demo/ui_assets/_jt_Group_Block.csv
+35 −0 res/preload/lca_demo/ui_assets/_jt_View_Group.csv
Loading