Nowadays having a proper security configuration is a huge challenge, especially looking at the global hacks and personal data leak incidents that happened in IT a while back. Oracle EBS is not perfect and has lots of vulnerabilities covered by Oracle almost every quarter. A very small percent of Apps DBAs know all the features and options available, and usually, do not go over firewall/reverse proxy layer.
This presentation is going to cover an overview and recommendations of options and security features that are available and can be used out-of-the-box, and some of the non-trivial configurations that can help to keep your Oracle EBS system protected, per our experience.
Uncommon Grace The Autobiography of Isaac Folorunso
Security of Oracle EBS - How I can Protect my System (UKOUG APPS 18 edition)
1. Security of
Oracle E-Business Suite
How I can protect my system
ANDREJS PROKOPJEVS
Lead Applications Database Consultant
2. About Andrejs
2
Apps DBA from Riga, Latvia.
Speaking SQL since 2001. In Oracle world since 2004.
Boiling Oracle EBS since 2006.
Conference speaker:
UKOUG, nlOUG, DOAG, OAUG Collaborate
UKOUG 2017 Speaker Award winner
Andrejs Prokopjevs
Lead Applications Database Consultant
At Pythian since 2011
@aprokopjevs
prokopjevs@pythian.com
https://www.pythian.com/blog/author/prokopjevs/
6. Agenda
• Overview and General Processes
• Servers and OS
• Database
• Applications Techstack
• Applications
• Something Extra
6
7. Overview
• Why we are talking about this?
• Data security became one of the major areas.
• New generation of compliance requirements.
• Cloud.
• Too many resonance breaches went into public in the past few years.
• Majority of engineers still don’t pay the attention needed.
• Missing skills.
• Presentation scope is Technical. Functional area – very limited overview.
7
8. General Processes
• Be on top of latest security alerts and updates
• Quarterly release by Oracle for most of the products and components.
• Once alert is released public it becomes even more critical to patch the
vulnerability ASAP.
• Keep your software up-to-date
• Everything, not just Oracle.
• Monitor system activity
• Not just system up status, but also who/what is using the system now.
• What is being used, usage trends by components.
8
9. General Processes #2
• Audit levels vs system overhead
• High audit data collection is impacting system performance.
• Need to look for a “golden middle” per requirements.
• Principle of least privilege
• Don’t assign permissions more than a role requires.
• Test instances and developer access to production data
• Control who access your production data, especially from third party.
• Obfuscate sensitive data in Test and Development during your cloning process
9
10. General Processes #3
• Restrict the access to a production segment
• Control who and how access your production data.
• Add extra protection layer for maintenance activities.
• Terminate direct external VPN access segment
• At least, for external consultants and third party.
• Replace with terminal access to avoid BYOD risks.
• Tehama – https://www.tehama.io/
10
11. General Processes #4
• We don’t care or think that all this is not important for us until we become
the target of the attack
• Prevent proactively – system and data breach can cost a lot !
11
13. Servers and OS – Access
• Use personal accounts and elevate access using su or sudo.
• Restrict direct root access from outside world completely.
• Restrict direct oracle and applmgr access from outside world.
▪ Exception - system runtime requirements (R12.2).
• Secure and control personal accounts who can access and what.
• Integrate external LDAP for easier access control management.
• Audit.
13
14. Servers and OS – Access #2
• Some maintenance and operational tasks may require additional non-
SRE/DBA (developer) access.
• Use dedicated accounts created with chroot-ed restricted shell or just sftp and
allow only what is required for specific task.
• Audit and record the terminal trail.
• R12.2: Majority of developer tasks (ex recompilations) do not work
anymore with same group non-applmgr access.
• Elevate access via sudo, but only for non-production instances.
• Apply wrappers and allow sudo only for specific wrapper executions. Protect
these wrappers well.
14
15. Servers and OS – Access #3
• Cleanup and control file and directory ownership and access.
• Avoid read/write/execute permissions for others unless having a strong
requirement. Do a cleanup.
• Set umask for the runtime to create new content with proper permissions. Actual
for interfacing endpoints too (like SFTP). Requires constant watch and analysis
since EBS has bugs/patches and there might be processes ignoring or even
overriding your umask value set.
15
16. Servers and OS – Network
• A network firewall is restricting connections from outside world.
• Configure local server firewall service to control the local subnet.
• Plus help the network firewall as a 2nd level of defense.
• Audit.
16
18. Database – Security patching
• SPU patch, also included in DB PSU or ProactiveBP.
• Recommended, but, usually, not critical. Often contains low priority non-remote
vulnerability fixes for components like Oracle Text, XDB, etc.
• Each CPU release has to be reviewed for the content details.
• R12.2 - ETCC certification covers last 2-3 PSU only.
• Keep OJVM up to date.
• Constantly there are 9+ CVE score remote exploits covered every quarter.
• Used by FND authentication core (dbc, password encryption/decryption).
• Not depended on DB PSU level (min Oct2014 - 11.2.0.4.4 or 12.1.0.2.1).
18
19. Database – Secure TNS listeners
• SQL*Net traffic encryption
• Default unencrypted listener controlled by AutoConfig.
▪ Used by Oracle Forms 10.1.2.3 (limitation).
▪ Enable Invited Nodes via FND_SQLNET_ACCESS profile option.
▪ Protect on the network level.
▪ Will run the EXTPROC service isolated.
• Custom TCPS enabled listener (one-way TLS server authentication).
▪ Used by all other EBS runtime, and only by EBS runtime.
▪ Configure and replicate the Invited Nodes.
▪ Protect on the network level.
• Custom “public” TCPS enabled listener for external DB access.
▪ Use case: DBAs, Interfaces, Developers.
▪ Control who can access on the network level. Audit.
19
20. Database – Secure TNS listeners #2
• More on “public” TNS listener
• Implement TCPS with two-way TLS client-server authentication.
▪ More secure. Source is allowed only if client certificate is in place.
▪ Control who and how long can access the instance using the certificate expiration issued.
• Optionally an external Connection Manager can be used as an extra layer protection.
• Class of Secure Transport (COST)
• Restricts remote instance registration.
• ADMIN_RESTRICTIONS=ON (s_admin_restrictions)
• Restricts remote configuration updates via lsnrctl.
• CONNECT_TIMEOUT_X
• Set a reasonable connection establishment time limit.
20
21. Database – Initialization parameters
• Basic recommendations
• REMOTE_OS_AUTHENT=FALSE and REMOTE_OS_ROLES=FALSE
• _TRACE_FILES_PUBLIC=FALSE
• O7_DICTIONARY_ACCESSIBILITY=FALSE
• UTL_FILE_DIR
• Control the value. By default only APPLPTMP requires it.
21
22. Database – Passwords
• Default passwords
• select * from DBA_USERS_WITH_DEFPWD;
• SEC_CASE_SENSITIVE_LOGON=TRUE
• Set by default to false, even with R12.2.
• R12.2 – no patching required, R12.1 – Patch 12964564.
• Only APPLSYSPUB requires an exception – ALL UPPERCASE password.
• Reference: Doc ID 1581584.1
• Lock all non-standard-EBS schemas (not part of
FND_ORACLE_USERID) that are not in use.
22
23. Database – Profiles
• Create, at least, three type of profiles:
• 1 - Application runtime profile
• Used for application runtime schemas including all registered under
FND_ORACLE_USERID. Should have soft settings, like
PASSWORD_LIFE_TIME set to UNLIMITED.
• Can also be split into two separate profiles:
▪ General with FAILED_LOGIN_ATTEMPTS control.
▪ Dedicated for APPS and APPLSYSPUB with unlimited FAILED_LOGIN_ATTEMPTS to
avoid target crash attacks.
• Audit the invalid authentication attempts.
• Ensure sec_max_failed_login_attempts init parameter is not set to unlimited and
has reasonable value from 1 to 3 (default 10).
23
24. Database – Profiles #2
• Create, at least, three profiles (continued):
• 2 - Administrator profile
• Used for SYS, SYSTEM and other administrative schema access. Should have
strong settings with good password rotation process in place.
• 3 - Public user profile
• Used for all other public access to the database. Should have strong settings
with good password rotation process in place with reasonably minimum
PASSWORD_LIFE_TIME value.
• Watch invalid login attempt trends to detect password scanning.
• Audit.
24
25. Database – Passwords #2
• Profiles and PASSWORD_VERIFY_FUNCTION
• Deploy your own function to control password complexity per requirements.
• Apply different functions for different profile levels. Application runtime profile has
limitations:
▪ FNDCPASS managed users (product schemas, APPS, APPLSYSPUB, etc) do not
support special characters. Reference: 1336479.1.
• Logon Trigger
• Audit.
• Control who and from what source can make a database connection.
• Limit runtime connections with weak protection to be allowed local only.
25
26. Database – Grant and access control
• Non-seeded object grants must be allowed only per requirements.
• Revoke all non-seeded *ANY* grants (like SELECT ANY TABLE).
• Avoid using them in the future.
• Protect product schema access similar to APPS.
• Many seeded schemas have grants like CREATE/DROP ANY PROCEDURE,
TRIGGER etc.
• Limit developer access also for custom product top schemas.
• Create proxy, or personal dedicated accounts and grant the privileges required
only.
• Enterprise User Security can be used to control access via global roles and proxy
schemas.
26
27. Database – Grant and access control #2
• Control and monitor permissions granted to APPLSYSPUB.
• SELECT * FROM dba_tab_privs WHERE grantee ='APPLSYSPUB’;
• Security Guide for any EBS release outlines the required set of permissions.
• Control your Database Links, especially incoming connections.
• Better to be isolated to a dedicated DB user with restricted permissions.
27
28. Database – Auditing
• AUDIT_SYS_OPERATIONS=TRUE is a minimum must.
• Use traditional database audit - AUDIT_TRAIL.
• Recommended to audit, at least, DML for tables like FND_USER and
WF_LOCAL_USER_ROLES.
• Security Guide for any EBS release outlines the minimum recommended object
audit list.
• Fine-Grained Auditing can also be used for more advanced scenarios.
• New 12c+ Unified Auditing is not supported with EBS.
• Use custom triggers to audit DDL operations.
28
29. Database – Other
• Storage data encryption – both TDE tablespace and column encryption
are supported with EBS
• Using TDE Tablespace Encryption with Oracle E-Business Suite Release 12
(Database 12c) (Doc ID 1584458.1)
• Using TDE Column Encryption with Oracle E-Business Suite Release 12.2 (Doc
ID 1585696.1)
• Fine grained access control – implement Database Vault.
• Integrating Oracle E-Business Suite Release 12.2 with Oracle Database Vault
12c (Doc ID 2131435.1)
• Oracle Database Security Assessment Tool (DBSAT) (Doc ID 2138254.1)
• Provides valuable general inputs, but has a lot of false-positive content for Oracle
EBS.
29
31. Applications Techstack – Security patching
• Each techstack component is covered separately.
• R12.1 - 10.1.3 / 10.1.2 are de-supported, but still major security issues are
covered exclusively for Oracle E-Business Suite.
• R12.2 - Same for 10.1.2. Other components are still under
premier/extended support umbrella till Dec 2021.
• R12.2 - ETCC is not covering all the security patches. Don't rely on ETCC
only.
• Often CPU release document for Fusion Middleware contains more fixes
for a component product than the one for E-Business Suite.
• Not all are applicable for EBS. Examples: WebCache or JDBC driver updates.
• Don't forget about your JDK. Quite often it is missed or ignored.
31
32. Applications Techstack – Reverse proxy
• Adds extra layer that hides the back-end – security recommendation.
• R12.1 - OHS 10.1.3.5 is based on Apache 2.0 that is “dead”.
• R12.2 - OHS 11.1.1.9 is based on Apache 2.2 which is in EOL since June
2017, but still covered by Oracle support.
• Can run the latest Apache 2.4 with all the latest security features turned
on protecting your environment that are not included with core EBS.
32
33. Applications Techstack – WebLogic in R12.2
• Protect WLS AdminServer from any external access
• All is because of T3 protocol which is weak and vulnerable.
• Restrict on network level.
• Hide under a Reverse Proxy and mod_weblogic.
• Best to restrict T3 protocol and allow only HTTP or HTTPS.
• Protect weblogic access.
• Personal accounts (local or SSO) can be implemented and used for daily use.
• Maintenance work still requires the local “weblogic” user.
33
34. Applications Techstack – Audit
• Standard Apache logging
• AccessLog / LogFormat
• Error/SSL/mod_security2 logs.
• Standard WebLogic logging
• Fusion Middleware Audit Framework (OPSS) is not in use by EBS.
34
36. Applications – Security patching
• Cumulative.
• Sometimes have cross-product dependencies.
• Sometimes have AD/TXK dependencies and delivers new configuration
updates.
• Often covers critical remote exploits with 9+ CVE score.
• Functional vulnerabilities that technical layer is not protecting.
36
37. Applications – Passwords
• Default passwords
• $FND_TOP/patch/115/sql/fnddefpw.sql
• EBSCheckUserPasswords.sql - Doc ID 2069190.1
• User password complexity control
• SIGNON_PASSWORD_% profile options
• Security User Define form (FNDSCAUS)
• FNDCPASS has limitations supporting special characters. Reference: 1336479.1
37
38. Applications – Passwords #2
• User password complexity control (continued)
• Signon Password Case (SIGNON_PASSWORD_CASE).
▪ Case sensitivity for passwords.
• Signon Password Custom (SIGNON_PASSWORD_CUSTOM).
▪ Custom java class to deploy your own control for password complexity per requirements.
• Signon Password Failure Limit (SIGNON_PASSWORD_FAILURE_LIMIT).
▪ Max number of unsuccessful login attempts before the lockout.
• Signon Password Hard To Guess (SIGNON_PASSWORD_HARD_TO_GUESS).
▪ Enables password requirements: 1) at least one letter and at least one number 2) doesn’t contain
username 3) doesn’t contain repeating characters.
• Signon Password Length (SIGNON_PASSWORD_LENGTH).
▪ Minimum length of a password.
• Signon Password No Reuse (SIGNON_PASSWORD_NO_REUSE).
▪ Number of days before reusing an earlier used password. 38
39. Applications – Passwords #3
• User password complexity control
(continued)
• Security User Define form (FNDSCAUS).
▪ Password expiration.
▪ Days – password lifetime.
▪ Accesses – how many times
▪ None – no expiration.
▪ Password expiration is handled on a user level.
There is no centralized control.
39
40. Applications – Passwords #4
• User password complexity control (continued)
• Very limited functionality not meeting today’s requirements.
• Implement Signon Password Custom (SIGNON_PASSWORD_CUSTOM)
• Implement Oracle Access Manager
▪ Modern password policy support
▪ Options for Multi Factor Authentication implementation.
• Password Policies in Oracle Access Manager (UKOUG Apps 16)
40
41. Applications – Passwords #5
• Deprecated product accounts should be disabled
• If upgraded from previous releases
• Reference Security Guide for your release
• Monitor and control Unsuccessful Login Attempts
(FND_UNSUCCESSFUL_LOGINS/ICX_FAILURES)
• Detect scans.
• Partially applicable if SSO is implemented. Covers only local authentication.
41
42. Applications – Passwords #6
• Set maximum and idle user session limits
• ICX:Session Timeout (ICX_SESSION_TIMEOUT)
• ICX: Limit time (ICX_LIMIT_TIME)
• Implement non-reversible password hashing (min SHA-256)
• Patch 26175708 FND SECURITY RUP JUN-2017
• Enhance Security With Non-Reversible Hash Password FNDCPASS
(Doc ID 457166.1)
42
43. Applications – Technical Configuration
• Implement SSL (TLS 1.2)
• Minimum – SSL termination on Reverse Proxy or Load Balancer.
• Recommended – full native SSL implementation on all layers.
• Java Code Signing certificate implementation for Forms
• Move away from default UNKNOWN signed jar files. Reference: 1591073.1.
• Keep client side JRE up-to-date
• Requires SRE involvement.
• Force minimum Java plugin requirements in configuration.
• External tiers have to be in DMZ configuration and properly restricted
url_fw (URL firewall).
43
44. Applications – Technical Configuration #2
• FND: Connection Tagging (FND_CONNECTION_TAGGING)
• Maps DB session information with current Apps module and activities.
• Ensure and control that it is always enabled.
• Cookie Domain Scoping (ICX_SESSION_COOKIE_DOMAIN)
• Enforces the session cookie domain setting.
• Host – ideal for External. Domain – ex. Multi Tier / SSO integration.
• Custom – old fashioned approach with ICX_PARAMETERS.
• Clickjacking Protection - R12.2 enabled by default
• httpd.conf - Header set X-Frame-Options SAMEORIGIN
• Protects EBS from running as a frame or sub-object. Insecure approach.
44
45. Applications – Technical Configuration #3
• Stop using APPS credentials with adcmctl.sh/adstpall.sh/adstrtal.sh
• Created dedicated FND user with "Concurrent Manager Operator” responsibility.
• Update s_cp_user/s_cp_password_type.
• Instead use “-secureapps” parameter.
• R12.2: You still need to supply WebLogic admin password.
• Ensure s_appserverid_authentication=SECURE
• Only FND_NODES registered sources can make a connection.
• Ensure s_admin_ui_access_nodes is restricted to localhost or limited
admin hosts
• trusted.conf - Protect Administrative Web Pages
45
47. Applications – Allowed Resources
• Allowed Resources
• R12.2.6 + Patch 24737426:R12.FND.C.
• “Security: Allowed Resources” profile option
(FND_SEC_ALLOWED_RESOURCES) set to CONFIG.
• Functional Administrator / Allowed Resources.
• Analyze your access_log and allow only what is required.
• Whitelist JSPs and Servlets lists.
• Limitations – Function based pages like OA.jsp or RF.jsp.
▪ Example: OA.jsp?OAFunc=OAHOMEPAGE
47
48. Applications – Functional Configuration
• Use shared responsibilities instead of shared accounts
• Delegation (or impersonalization) – proxy users with enabled audit
• Least privilege approach
• Role based access control
• Review and limit responsibilities and permissions
• Data Security Policies
• Workflow
• WF_ADMIN_ROLE should not be “*”
• WF Mailer: Set SEND_ACCESS_KEY to N – Notification Details link in the email
requires a full authentication.
48
49. Applications – Functional Configuration #2
• Safe Authentication by Concurrent Managers
• Example: XXXX.prog APPS/APPS <other parameters>
• Prevents Forms enter-query feature.
• Execution Options define parameter field:
▪ ENCRYPT – APPS credentials will be encrypted.
▪ SECURE – APPS credentials are not passed. Use this option, if not needed.
• Attachment upload
• FND_SECURITY_FILETYPE_RESTRICT_DFLT=Y – protects only from Windows executable
content. If N – all is restricted except that is allowed by
FND_MIME_TYPES.ALLOW_FILE_UPLOAD column value.
• UPLOAD_FILE_SIZE_LIMIT – Set the maximum upload file size (Kb).
• FND_DISABLE_ANTISAMY_FILTER=Y – Filters the HTML content of attachment and
protects from malicious scripting.
49
51. Applications – Secure Configuration Console #2
• Has a collection of cross-suite security checks.
• R12.2.7+ post upgrade mandatory step. System is in Lockout mode until all tests
are passed or suppressed.
• Can be called at any point of time.
• Manual call: $ADJVAPRG oracle.apps.fnd.security.AdminSecurityCfg
• All checks are also available as external scripts (R12.1, < R12.2.7)
• Security Configuration and Auditing Scripts for Oracle E-Business Suite (Doc ID
2069190.1)
• EBSSecConfigChecks.zip
• EBSAuditScripts.zip
• Included in Application Management Pack.
51
52. Applications – Audit
• Standard minimum audit for data changes
• Tables like FND_USER or FND_PROFILE_OPTION_VALUES.
• Who columns: LAST_UPDATE_DATE, LAST_UPDATE_BY and
LAST_UPDATE_LOGIN.
• FND debug enabled with default level "Unexpected”
• Sign-On Audit
• Sign-On:Audit Level: None/User/Responsibility/Form
▪ Populates login data to FND_LOGINS, FND_LOGIN_RESPONSIBILITIES, and
FND_LOGIN_RESP_FORMS respectively.
• Sign-On:Notification: Y
▪ Notifies end user about unsuccessful login attepmts.
52
53. Applications – Audit #2
• Page Access Tracking
• Once enabled, writes data to a stage table first.
• Unavailable for reporting. Requires “Page Access Tracking Data Migration”
concurrent program to be scheduled.
• FND: Connection Tagging (FND_CONNECTION_TAGGING)
• Audit Trail
• Duplicates the Audit Trail feature of the database. Database audit tracks a more
detailed record change trail from database perspective, but missing key columns
that makes it an easy process to link FND login session to the data change.
• Audit Trail:Activate=Yes
53
54. Applications – Audit #3
• Audit Trail (continued)
• Auditing Manager / Audit Trail
• Install – enables the trail for registered DB schemas.
• Groups – enables the trail for particular tables and groups of tables.
• Limited configured objects out-of-the-box (FND_PROFILE_OPTION_VALUES is
included).
• Security Guide outlines the recommended list to be configured (FND, WF, others).
54
56. Applications – Role based separation of the Apps tiers
• Restrict Responsibilities by Web Server Trust Level
• Administrative
• Normal
• External
• It’s possible to isolate System and Functional Administration into a
separate protected apps tier.
56
57. DevOps and Automation
• Protect your configuration and policies.
• Central management.
• Ensure nobody overwrites your current configuration.
• Detect and alert about unplanned changes.
57
58. mod_security / mod_rewrite customizations
• Add your own additions and content filters per your security requirements
• R12.2: Customize your templates
• Reverse Proxy: Ensure only Reverse Proxy hosts can connect to your
Applications on a network level
• Can extend your Allowed Resources filters
• First level Gateway.
• URL gateway for Internal tier.
• Workarounds the Function limitation. Add additional regexp conditions and filter
requests like OA.jsp?OAFunc=OAHOMEPAGE.
58
59. web.xml
• What if we don’t trust mod_rewrite or Alowed Resources?
• What if we want 100% assurance that unused modules/pages won’t work?
• $EBS_APPS_DEPLOYMENT_DIR/oacore/html/WEB-INF/web.xml
• $FND_TOP/admin/template/oacore_web_xml_FMW.tmp
• Apply your own web.xml (automation) or customize the AutoConfig template.
• Remove JSPs and Servlets that you don’t want to run on a back-end.
• Similar limitation – can’t filter OA/RP.jsp functions.
59
60. Database – Privilege Elevation
• PL/SQL objects with AUTHID DEFINER
• Each product schema has APPS_DDL and APPS_ARRAY_DDL packages.
▪ Most of the APPS_DDL have grants for APPS except OKC has execute permission to call
APPS.APPS_DDL.
▪ APPS_ARRAY_DDL are targeted by SYSTEM only (AD_DDL and AD_INVOKER via
adadmin).
• APPS API PL/SQL objects, including FND_PROFILE and FND_SECURITY_PKG
packages.
▪ Watch the custom users/schemas carefully whom you are granting direct execute
permissions.
• SYS.AD_GRANTS
▪ Only APPS and SYSTEM have execute permissions. SYSTEM with grant option.
▪ Recent security updates limited the API with only REVOKE_GRANT() function.
60
61. Database – Privilege Elevation #2
• PL/SQL objects with AUTHID DEFINER (continued)
• R12.2: SYS.AD_ZD_SYS
▪ Only APPS and SYSTEM have execute permissions. SYSTEM with grant option.
▪ Recent security updates limited the API with functions like drop and actualization of the
editions, or enable/disable the R12.2 logon trigger (call from ad_zd.alter_logon_trigger API).
• SYS.DBMS_SYS_SQL
▪ Part of the addb1010.sql/addb1020.sql/audb1120.sql
▪ Only SYSTEM has execute permissions with grant option.
▪ Required for SYSTEM.AD_APPS_PRIVATE.copy_huge_view() – replaced with DBMS_SQL.
▪ Required for WebDB support – not in use anymore. Legacy functionality.
• SYSTEM.AD_DDL
▪ All product schemas have execute permissions.
▪ Execute is granted to PUBLIC. Fixed in recent CPU security updates for EBS. 61
62. Database – Privilege Elevation #3
• SYSTEM account permissions
• Has a lot of extra permissions in EBS database.
• CREATE/ALTER/DROP ANY *
• EXECUTE ANY PROCEDURE
• ADMINISTER DATABASE TRIGGER
• GRANT ANY PRIVILEGE/ROLE
62
64. Summary
• This presentation contains only ~40% of security content applicable to
Oracle E-Business Suite.
• This presentation contains 36 mentions of the word “Audit”. Know what is
happening in your system.
• Find a balance between your audit levels and system performance
overhead. End users and business must not suffer.
• All this, usually, is not important until we become the target of the attack.
Prevent proactively.
• Protect your privileged access at all cost.
64
