Ncep provider by AaronV77 · Pull Request #34 · erdc/quest

AaronV77 · 2018-04-09T21:41:17Z

Added the ncep provider, updated the docs, and added ncep_client and terrapin to the environment files for python 2 and 3.

…pin.

stickler-ci · 2018-04-09T21:41:27Z

+    display_name = 'NCEP GFS Provider'
+    description = 'Services avaliable through the NOAA NCEP Server.'
+    organization_name = 'National Centers for Environmental Prediction'
+    organization_abbr = 'NCEP'


W292 no newline at end of file

stickler-ci · 2018-04-09T21:41:27Z

 'svc://cuahsi-hydroshare:hs_geo': {},
 'svc://cuahsi-hydroshare:hs_norm': {},
+ 'svc://noaa-ncep:ncep_gfs': {'properties': [{'default': None,
+    'description': 'YYYYMMDD',


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-09T21:41:27Z

 'svc://cuahsi-hydroshare:hs_norm': {},
+ 'svc://noaa-ncep:ncep_gfs': {'properties': [{'default': None,
+    'description': 'YYYYMMDD',
+    'name': 'date',


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-09T21:41:27Z

+ 'svc://noaa-ncep:ncep_gfs': {'properties': [{'default': None,
+    'description': 'YYYYMMDD',
+    'name': 'date',
+    'type': 'String'},


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-09T21:41:27Z

+    'description': 'YYYYMMDD',
+    'name': 'date',
+    'type': 'String'},
+   {'default': None,


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-09T21:41:27Z

+    'description': 'Paramerter',
+    'name': 'format',
+    'range': [['enkf', 'enkf'],
+     ['gdas', 'gdas'],


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-09T21:41:28Z

+    'description': 'Parameter3',
+    'name': 'product',
+    'range': [["0.50 deg 'full' file description",
+      "0.50 deg 'full' file description"],


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-09T21:41:28Z

+    'name': 'product',
+    'range': [["0.50 deg 'full' file description",
+      "0.50 deg 'full' file description"],
+     ['32km Lambert Conformal grid', '32km Lambert Conformal grid'],


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-09T21:41:28Z

+     ['global longitude-latitude grid (1.0 deg)',
+      'global longitude-latitude grid (1.0 deg)']],
+    'type': 'ObjectSelector'}],
+  'title': 'NCEP GFS Service Download Options'}


E121 continuation line under-indented for hanging indent

* Added the cuahsi coverage fix. The change was not getting all the resources and just getting all resources that had a coverage from the world. * Made kwargs the standard over using options, params, and kwargs. * Updated the load_providers to only update if update-cache is True or if the global is None.

…pin.

…he tests.

stickler-ci · 2018-04-16T18:14:13Z


-    providers = {name: driver.DriverManager('quest.services', name, invoke_on_load=True, invoke_kwds={'name': name}).driver for name in web_services}
+    if update_cache or the_providers is None:
+        providers = {name: driver.DriverManager('quest.services', name, invoke_on_load=True, invoke_kwds={'name': name}).driver for name in web_services}


E501 line too long (153 > 120 characters)

stickler-ci · 2018-04-16T18:14:13Z

+        if len(settings.get('USER_SERVICES', [])) > 0:
+            for uri in settings.get('USER_SERVICES', []):
+                try:
+                    drv = driver.DriverManager('quest.services', 'user', invoke_on_load=True, invoke_kwds={'uri': uri}).driver


E501 line too long (126 > 120 characters)

…sts.

stickler-ci · 2018-04-16T20:01:40Z

+
+        return metadata
+
+class NCEPProvider(ProviderBase):


E302 expected 2 blank lines, found 1

stickler-ci · 2018-04-17T13:34:30Z

+      'global longitude-latitude grid (1.0 deg)']],
+    'type': 'ObjectSelector'}],
+  'title': 'NCEP GFS Service Download Options'},
+  'svc://noaa-ncep:ncep_nam': {'properties': [{'default': None,


E131 continuation line unaligned for hanging indent

stickler-ci · 2018-04-17T13:34:30Z

+    'type': 'ObjectSelector'}],
+  'title': 'NCEP GFS Service Download Options'},
+  'svc://noaa-ncep:ncep_nam': {'properties': [{'default': None,
+    'description': 'YYYYMMDD',


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-17T13:34:30Z

+  'title': 'NCEP GFS Service Download Options'},
+  'svc://noaa-ncep:ncep_nam': {'properties': [{'default': None,
+    'description': 'YYYYMMDD',
+    'name': 'date',


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-17T13:34:30Z

+  'svc://noaa-ncep:ncep_nam': {'properties': [{'default': None,
+    'description': 'YYYYMMDD',
+    'name': 'date',
+    'type': 'String'},


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-17T13:34:30Z

+    'description': 'YYYYMMDD',
+    'name': 'date',
+    'type': 'String'},
+   {'default': None,


E128 continuation line under-indented for visual indent

stickler-ci · 2018-04-17T13:34:30Z

+      'NAM 215 AWIPS Grid - CONUS (20-km Resolution)'],
+     ['NAM 216 AWIPS Grid - Regional - Alaska (45-km Resolution)',
+      'NAM 216 AWIPS Grid - Regional - Alaska (45-km Resolution)'],
+     ['NAM 218 AWIPS Grid - CONUS (12-km Resolution; full complement of pressure level fields and some surface-based fields)',


E501 line too long (126 > 120 characters)

stickler-ci · 2018-04-17T13:34:31Z

+     ['NAM 216 AWIPS Grid - Regional - Alaska (45-km Resolution)',
+      'NAM 216 AWIPS Grid - Regional - Alaska (45-km Resolution)'],
+     ['NAM 218 AWIPS Grid - CONUS (12-km Resolution; full complement of pressure level fields and some surface-based fields)',
+      'NAM 218 AWIPS Grid - CONUS (12-km Resolution; full complement of pressure level fields and some surface-based fields)'],


E501 line too long (127 > 120 characters)

stickler-ci · 2018-04-17T13:34:31Z

+      'NAM 221 AWIPS Grid - High Resolution North American Master Grid (32-km Resolution)'],
+     ['NAM 221 AWIPS Grid - N. American Master (32-km Resolution) (GOES Simulated Brightness Temp.)',
+      'NAM 221 AWIPS Grid - N. American Master (32-km Resolution) (GOES Simulated Brightness Temp.)'],
+     ['NAM 242 AWIPS Grid - Over Alaska (11.25 KM Resolution; full complement of pressure level fields and some surface-based fields)',


E501 line too long (135 > 120 characters)

stickler-ci · 2018-04-17T13:34:31Z

+     ['NAM 221 AWIPS Grid - N. American Master (32-km Resolution) (GOES Simulated Brightness Temp.)',
+      'NAM 221 AWIPS Grid - N. American Master (32-km Resolution) (GOES Simulated Brightness Temp.)'],
+     ['NAM 242 AWIPS Grid - Over Alaska (11.25 KM Resolution; full complement of pressure level fields and some surface-based fields)',
+      'NAM 242 AWIPS Grid - Over Alaska (11.25 KM Resolution; full complement of pressure level fields and some surface-based fields)'],


E501 line too long (136 > 120 characters)

stickler-ci · 2018-04-17T13:34:31Z

+     ['global longitude-latitude grid (1.0 deg)',
+      'global longitude-latitude grid (1.0 deg)']],
+    'type': 'ObjectSelector'}],
+  'title': 'NCEP GFS Service Download Options'},


E121 continuation line under-indented for hanging indent

* I got the basic authentication piece setup for the nasa provider. * Made a small fix to the authentication fix. This provider is working again. * Nasa provider syntax fix. * Minor updates (#38) * Added the cuahsi coverage fix. The change was not getting all the resources and just getting all resources that had a coverage from the world. * Made kwargs the standard over using options, params, and kwargs. * Updated the load_providers to only update if update-cache is True or if the global is None. * I got the basic authentication piece setup for the nasa provider. * Made a small fix to the authentication fix. This provider is working again. * Nasa provider syntax fix. * I never added the uncommented the nasa download options to the tests in data.py. This fixes the tests.

…pin.

…he tests.

…sts.

# Conflicts: # quest/services/noaa_ncep.py

stickler-ci · 2018-04-19T16:48:03Z

+
+    def _read_granules(self, short_name, page_num):
+        try:
+            return requests.get(granules_url % (short_name, page_num), auth=(self.info['username'], self.info['password'])).json()['feed']['entry']


E501 line too long (147 > 120 characters)

stickler-ci · 2018-04-19T16:48:03Z

+    display_name = 'NCEP Provider'
+    description = 'Services avaliable through the NOAA NCEP Server.'
+    organization_name = 'National Centers for Environmental Prediction'
+    organization_abbr = 'NCEP'


W292 no newline at end of file

stickler-ci · 2018-04-19T16:48:03Z

 SERVICE_FEATURE_DOWNLOAD_OPTIONS = [
-    # ('svc://nasa:srtm-3-arc-second/G1034711987-LPDAAC_ECS' , None),
-    # ('svc://nasa:srtm-30-arc-second/G1005651728-LPDAAC_ECS', None),
+    ('svc://nasa:srtm-3-arc-second/G1034711987-LPDAAC_ECS' , None),


E203 whitespace before ','

stickler-ci · 2018-04-19T16:48:04Z

 DOWNLOAD_OPTIONS_FROM_ALL_SERVICES = {
- # 'svc://nasa:srtm-3-arc-second': {},
- # 'svc://nasa:srtm-30-arc-second': {},
+ 'svc://nasa:srtm-3-arc-second': {},


E121 continuation line under-indented for hanging indent

stickler-ci · 2018-04-19T16:50:46Z

+    'description': 'Parameter3',
+    'name': 'product',
+    'range': [['NAM', 'NAM'],
+     ['NAM - Binary Universal Form for the Representation of meteorological data (BUFR)',


E128 continuation line under-indented for visual indent

…for getting the status of a providers authentication. (#41)

…pin.

…he tests.

…sts.

sdc50

Tests failing because of the NASA provider, which will be fixed in a future PR.

AaronV77 and others added 5 commits April 9, 2018 16:33

Dataset display name fix (More readable).

37838a7

Added the noaa_ncep plugin and updated the conda environment file.

47b78dc

NCEP test and setup.cfg were updated.

e790d72

Removing the placeholder for the girder site.

00aa22c

Updated the docs and the python 2 environment file for ncep and terra…

5ff0e5e

…pin.

stickler-ci reviewed Apr 9, 2018

View reviewed changes

AaronV77 assigned AaronV77 and sdc50 and unassigned AaronV77 Apr 9, 2018

AaronV77 added the enhancement New feature or request label Apr 9, 2018

AaronV77 added this to the RD WorkShop milestone Apr 9, 2018

AaronV77 and others added 8 commits April 16, 2018 11:45

Dataset display name fix (More readable).

c488638

Added the noaa_ncep plugin and updated the conda environment file.

f9121ce

NCEP test and setup.cfg were updated.

ab0920b

Removing the placeholder for the girder site.

e48dab0

Updated the docs and the python 2 environment file for ncep and terra…

b586e50

…pin.

The setup.cfg never got updated or added to this commit. This fixes t…

f076882

…he tests.

Merge remote-tracking branch 'origin/ncep_provider' into ncep_provider

714f2b0

stickler-ci reviewed Apr 16, 2018

View reviewed changes

Addeed another service from ncep, and added the sorted call to the li…

649678e

…sts.

stickler-ci reviewed Apr 16, 2018

View reviewed changes

Fixed the test and added the sorted functions into this plugin.

0466a29

stickler-ci reviewed Apr 17, 2018

View reviewed changes

AaronV77 and others added 6 commits April 18, 2018 08:22

Dataset display name fix (More readable).

ff48887

Added the noaa_ncep plugin and updated the conda environment file.

6e7f1a8

NCEP test and setup.cfg were updated.

9b23c1b

Removing the placeholder for the girder site.

f40a67e

Updated the docs and the python 2 environment file for ncep and terra…

2ff755c

…pin.

AaronV77 added 6 commits April 19, 2018 11:34

Removing the placeholder for the girder site.

3906d87

The setup.cfg never got updated or added to this commit. This fixes t…

cb32c40

…he tests.

Addeed another service from ncep, and added the sorted call to the li…

34c3623

…sts.

Fixed the test and added the sorted functions into this plugin.

9133312

Fixed the test for ncep provider.

5e3f17e

Merge remote-tracking branch 'origin/ncep_provider' into ncep_provider

1f3106e

# Conflicts: # quest/services/noaa_ncep.py

stickler-ci reviewed Apr 19, 2018

View reviewed changes

Fixed the test for ncep provider part 2.

313d6d0

stickler-ci reviewed Apr 19, 2018

View reviewed changes

AaronV77 and others added 18 commits April 20, 2018 16:04

Updated the authentication to work with QuestWeb, and added a method …

d2af8ab

…for getting the status of a providers authentication. (#41)

Dataset display name fix (More readable).

3289b3e

Added the noaa_ncep plugin and updated the conda environment file.

059fec6

NCEP test and setup.cfg were updated.

47b067d

Removing the placeholder for the girder site.

fdb30cd

Updated the docs and the python 2 environment file for ncep and terra…

e928842

…pin.

Dataset display name fix (More readable).

fa8d7df

Removing the placeholder for the girder site.

a5efdef

The setup.cfg never got updated or added to this commit. This fixes t…

ba74079

…he tests.

Addeed another service from ncep, and added the sorted call to the li…

00bf610

…sts.

Fixed the test and added the sorted functions into this plugin.

f63a734

Dataset display name fix (More readable).

60e4ed3

Removing the placeholder for the girder site.

64a6240

Dataset display name fix (More readable).

9d0e1fb

Removing the placeholder for the girder site.

dddc5e9

Fixed the test for ncep provider.

a7fe179

Fixed the test for ncep provider part 2.

0be5eec

Merge remote-tracking branch 'origin/ncep_provider' into ncep_provider

290a85b

sdc50 approved these changes Apr 27, 2018

View reviewed changes

sdc50 merged commit 6a80777 into erdc:master Apr 27, 2018

AaronV77 deleted the ncep_provider branch May 8, 2018 17:10

Uh oh!

Conversation

AaronV77 commented Apr 9, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sdc50 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants