Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (6)
Ähnlich wie FME User Conference Presentation
Ähnlich wie FME User Conference Presentation (20)
FME User Conference Presentation
- 1. Data Validation using FME Server Clay Wise, OCP, GISP Enterprise Architect Hampton Roads Sanitation District
- 2. Abstract (Hidden Slide) The Hampton Roads Sanitation District (HRSD), which serves 17 localities in southeastern Virginia and a population of 1.7 million, has embarked on a Virginia Department of Environmental Quality (VDEQ) driven effort to upgrade its utility management systems in support of a regional hydraulic wastewater model. The hydraulic model must include the entire interceptor infrastructure as well as input from the 14 separate models of the participating localities that feed into the collection system. With data inputs coming from so many different sources, each with their own data schemas, a solution needed to be developed to manage the continuous transfer of data from the localities to the regional hydraulic model, as well as from HRSD back to the localities. Topics covered in this presentation will include how FME Server was customized and implemented to streamline the input and validation of the various data sources, how the solution provided tremendous time saving and data quality insurance, and how FME Server helped resolve complex data sharing challenges.
- 4. Quick Facts Treatment of Sanitary Sewer for over 1.7 million people. More than 500 miles of pipes, 6-66 inches in diameter 102 Pump Stations 9 Major Treatment plants 231 million gallons per day is treated Serving 17 cities/counties
- 5. Regulatory Requirements Context: Sanitary Sewer Overflows Task: Regional Hydraulic Model Mostly Pressurized System 14 Jurisdictional Data Inputs 14 Jurisdictional GIS Database Designs Numerous Data Inputs Over 1,800 jurisdictional pump stations Over 6,500 miles of jurisdictional sewer mains Over 120,000 sewer structures (manholes, vaults, etc…) Challenge: Data Sharing Jurisdictional Hydraulic Model and general GIS Data Inputs to HRSD HRSD Regional Hydraulic Model Data to Jurisdictions Data Validation & Standardization, Data Updates & Corrections
- 8. Specific values (material, locality code)
- 9. How many were correct out of the total
- 10. Is the data spatially correct?
- 11. Is the geometry valid?
- 13. How do you upload and download to FME Server? FME Workbench Desktop product to develop and publish data validation workspaces to FME Server Once published the workspace can be run through a web browser End users do not need an FME license to run the translation Can also download the posted workspace to make changes
- 14. How are we validating data? FME Workbench Desktop product to construct/develop the validation process Simple upload to publish process to FME Server Upload & Download FME Server
- 18. NULL values
- 21. Showing the results (continued..) <li class="error_message">cfgFailCount of cfgFeatCount features in your dataset have blank PSID attribute values.<ul> <li>The ids of the failed features are: cfgFailedIds.</li></ul></li><br/> cfg01PumpStations: Calls out a specific variable in the report web page.
- 22. Web page editing cfg01PumpStations: This is the same variable we called out in the previous slide. After the information goes through the validation process it is placed in this variable. You can change them to whatever name suits you as long as they match the between the workbench file, and the webpage. I use Dreamweaver (CS3), but you can use cheaper software such as UltraEdit to edit the HTML.
- 23. Web page editing Color coding the message displayed. Here I simply have red for an error, and green if all the features passed. cfg01Nodes: This is the same type of variable I have spoken about in the previous slides.
- 25. Green for passed, Red for failure
- 26. Connection points in blue are included with the summary checkscfgPipes02 cfgPipes03 cfgPipes04
- 29. Suffolk FME Server Example Suffolk will log in using OIM They will then find the GIS Parcels, and Water Meters that they maintain They will then run FME Server FME Server will join their GIS data to the water consumption data, and output a zip file for their use
- 31. Modelers provided feedback for the second run
- 32. Geometry Check – Complete
- 33. Some jurisdictions didn’t have very clean data (i.e. Self intersecting polygons, zero length lines)
- 34. Brief summary at the beginning of each feature class – Complete
- 35. Summary of linear feet of pipe- Complete
- 36. Connection point validation – Complete
- 38. Questions Questions? For more information: Clay Wise: cwise@hrsd.com HRSD: www.hrsd.com Data validation: fmepedia.com FME Server > F > FME Server Examples>D > Data Validation with FME Server
Hinweis der Redaktion
- Introduction: Who am I? Job role & certifications
- Who is HRSD? What is our purpose?What should come away with in this presentation?
- HRSD Mission Statement: to prevent pollution, protect public health, and support community development by providing wastewater collection, treatment, and related services.
- Quick Facts about HRSD.Just try to get the audience to understand the amount of sanitary sewer being treated.
- What are the regulatory requirements?How did HRSD and the Jurisdictions get to this point?How do we comply (meet deadlines) with the state orders/standards?What happens if we do not comply?CourtFines
- The Validation ToolWe are validating against a set of standards developed by HRSD and the Jurisdictions.They didn’t expect HRSD to validate against all of their data.Complete reporting of their modeling data.Explain connection points and why they are important.
- Explain the process before going into data validation with both feet. Try to at least show the FME Desktop process to upload during the demo.
- Very simple blow up of the beginning of one feature class being validated. Try to explain how I took the example workspace and started trying to conform it to what we needed. It was a great start. Had a lot of help from Safe Software, especially Aaron Koning.
- Common tester example.Validating against all the locality codes. They have to populate these so we know which jurisdiction’s data it is.
- Explaining briefly the group tester transformer that points to the html page. The results are shown based on the identifiers called out here.
- Quick shot of the HTML page with the variable name called out.
- Code example of the same html page. Just trying to show you can change the color, and other properties of the report to make it look any way you want.
- Example of the output. I want to show how the data is grouped based off of the modelers needs, not the jurisdictions. I also am trying to show how the cfg call outs are related to prior html page.
- Oracle Identity management. Currently we run the reports and provide them back to the jurisdictions. This will end in the near future with the implementation of OIM. We also have some other projects using FME Server and ArcGIS Server that need OIM to succeed. A good example is the City of Suffolk, and their need for billing data.
- City of Suffolk example of how we are trying to use FME Server to push a SOA architecture. Let the jurisdiction manage when they want the data.
- Feel very proud about the success we have accomplished during this project. We made sure to get a lot of feedback from the modelers. People are starting to ask for more FME Server apps to take care of processes.
- My contact information.