Marjorie M. K. Hlava, President, Chair of the Board, and Chief Scientist, Access Innovations, Inc.
During this annual highlight of the DHUG meetings, Margie will discuss the exciting new changes and additions to the Data Harmony software. She will be joined by some members of our software development team to talk about specific initiatives we have worked on over the past year.
1. Data Harmony Update
2021
v 3.15
The Access Innovations Development Team
Marjorie Hlava
mhlava@accessinn.com
2. 2020 Year of
the Covid
Plague
• Forced rapid change
• Changes in the way we operate
• Working form home
• New team supporting technology
• Smaller, more agile team
• Refocusing the software design list
• Back to basics
4. Then and Now
• Email threads Slack chat rooms
• Water cooler Daily Check In
• Conferences GoToMeeting, Zoom, Remo, Blue Jeans, Skype, etc.
• In the office Work from home
• End of Day Always on
• Clients do it themself Managed Services
• Bugs list JIRA Atlassian
• SVN Bit Bucket
• List of features desired Vetted planning modules
5. Massive changes on the inside
• Design meetings and plans
• By Module
• By Year
• Line by line code review
• Made each app modular
again
• Unique interfaces become
modules
• The Term base is fully a
database (again)
6. Big Lifts
• Stabilized the internal core
• The way things are parsed with in the system
• Memory usage is streaming not parsing as a block
• Means the data is moved smoothly through the system
• Rules act faster and more responsively
• More efficient loading
• Big O notation
• Increase “n” how long will it take to finish
• N square was high – now “n log n” time 2.8 instead of 10 x longer
8. Server Monitor
• Quick response due to some of our hosted servers
crashing / going offline without any way to reliable
way to restart them automatically or alert our
support team if they did go down.
• The monitor is be hooked up and managed by an
AWS database that it shares with the WebAdmin
monitor.
• The monitor will update the status of each server
in the AWS database as well as monitor any
projects in said database.
• If a server does go down, the monitor will attempt
to restart it up to 5 times.
• If all 5 restart attempts fail, the status of the server
will be updated in the AWS database and an alert
will be sent to Dataharmony and SWCP support.
(Command line project only, no screenshots)
9. Web Admin
• Better manage multiple hosted instances quickly
and reliably.
• This was designed as a Dataharmony In-house
project.
• It was built in conjunction with the Server Monitor,
and updates the AWS database with new server
instances, in turn the web admin will monitor any
projects that the Web Admin adds.
• From the Web admin, we can add/edit/remove Web
Admin users (Users who have access to use web
admin),
• modify many key Dataharmony servers and projects.
• Including editing admin password of the
server
• modifying settings of projects,
• project user management and more.
• No destructive operations exist on the web admin,
such as project deletion.
• For safety These processes still need to be
done manually with the default client or by
hand in the file system.
15. JIRA
• Jira is a cloud server
• Client support and project management platform
• In this platform you can create any number projects
• Set up individual project workflow schemas, and much more
• For each of these projects you can create issues, assign issues to staff
members, and much more
• Investigating using this for the client support system
• Software bug, Feature request, and more
• Connected to any web admin or linked to directly
• Clients can submit support issues and requests directly to us via Jira
• Includes automatic email responses
• Our wiki acts as our own internal Confluence
19. BitBucket
• Project management platform
• Developers publish code changes to our core
applications
• Developers then get the new code changes
anytime they wish
• Keep local code up to date
• Increased workflow efficiency
• Allows us to retain any and all change logs
• Ability to revert certain pieces of the code as
needed
20.
21. Improved Smart Submit
• Authors to submit their
articles
• Integrated publishing
workflow.
• Smart Submit automatically
indexes the submitted articles
• Key works from multiple
taxonomies
• Mike Millar will go into more
details
22. Client and General Improvements:
• Find terms missing rules
• Via MAI Rule Editor
• Find all complex rules
• Via MAI Rule Editor
• Find duplicate terms
• Via Thesaurus Master
• Importing stability is now greatly improved
• Code obfuscation
• This will require some training on how to correctly reverse a stack-trace,
since most of the code will now be highly obfuscated
• Admin panel adding/removing users bugs fixed
• Admin panel padding characters bug fixed
23. Server Stability Improvements
• Greatly increased the cache size for rules
• Improved stability for processing rules
• Larger volumes of text would occasionally cause
parsing to fail
• Server can now handle more concurrent
connections
• Database can now handle larger file sizes (4+ gigs)
30. Font Detection
• Beta of italics detection to
use in rule building…
• Used in the TaxoGene
taxonomy
• Basic change to the MAI
Engine
• Deep code changes.
• Use in a rule –
• If Bold, if underlined, if
italics –
• Do all three
• Use in combination
• Gabe is testing (Need
screen shot)
31. Picklist Field Attributes
• Data Harmony 3.15 introduces
picklist field attributes. Rather than
rely on free-text entries for simple
fields and text fields, users can
create multiple fields based on
created or imported picklists. These
provide additional facets for adding
more characteristics to term nodes.
The fields and picklists are added
through the DH Admin module and
can be selected in a drop-down list
within Thesaurus Master.
33. URL Hyperlinking to Web Browsers
Another field type added
to thesaurus terms
includes URL fields.
URL fields are meant to
link directly to websites
or webspaces which
contain a URL or a URI.
36. Check Terms Report
In MAI, a reporting system
has been added to check
for terms that are not
mentioned within any of
the system’s rules.
From the Search drop
down menu, select
Missing rules option.
Quick response project due to some of our hosted servers crashing / going offline without any way to reliable way to restart them automatically or alert our support team if they did go down.
The monitor is be hooked up and managed by an AWS database that it shares with the WebAdmin monitor.
The monitor will update the status of each server in the AWS database as well as monitor any projects in said database.
If a server does go down, the monitor will attempt to restart it up to 5 times.
If all 5 restart attempts fail, the status of the server will be updated in the AWS database and an alert will be sent to Dataharmony and SWCP support. (Command line project only, no screenshots)
Data Harmony 3.15 introduces picklist field attributes. Rather than rely on free-text entries for simple fields and text fields, users can create multiple fields based on created or imported picklists. These provide additional facets for adding more characteristics to term nodes. The fields and picklists are added through the DH Admin module and can be selected in a drop-down list within Thesaurus Master.
When a valid URL has been added to the field in Thesaurus Master, users can double-click the link to open up the website in their default browser. In the image below, double-clicking the clink takes the user to https://en.wikipedia.org/wiki/Social_science.
MAI Rulebuilder will generate a window which displays a list of terms not contained within a single rule in the thesaurus. This mode allows users to check for terms either accidentally removed from the rulebase (but not the thesaurus) and to checks for terms which were intentionally removed. The export feature allows the user to save the list as a single text file (txt) in order to share with a thesaurus development team.
Another feature added to the MAI Rulebuilder pane includes the “Get complex rules” search. This search feature includes a list of all compound rules for users to evaluate easily. The sub-window matches the same format as the Missing terms search, but the users can click on any rule in the list to immediately pull up the rule within the Rulebuilder pane. Users can export the list as a simple text field