ColdFusion AdminAPI to Remove Debugging IP Addresses

I just thought I would post a script to remove all the IP addresses inside the ColdFusion Admin.


You will need to change the password.

<cfset    adminObj = createObject("component","cfide.adminapi.administrator") />
<cfset    adminObj.login("YOURPASSWORDHERE") />
<cfset    myObj = createObject("component","cfide.adminapi.debugging") />
<cfset iplist = myObj.getIPList() />
Starting List
<cfdump var="#myObj.getIPList()#" />
<!--- if list is greater than 1 delete all IPs --->
<cfif listlen(iplist) gt 0>
<cfloop list="#iplist#" item="i" >
<cfoutput> Deleting #i#</cfoutput> <br />
<cfset myObj.deleteIP(i) />
</cfloop>

</cfif>
Ending List
<cfdump var="#myObj.getIPList()#" />

Nessus Security Scans can cause ColdFusion CPU spike

If your organization uses Nessus for security scans you might see an issue around the the jetty service ColdFusion uses for the PDFG Service.

Some ColdFusion accounts are reporting the same behavior after one of the scans is run against a CF server.  By looking at the IIS or Apache logs you can see clear indications that hundreds of erroneous requests made to the web server using to try and gain any penetration to the server.

Part of the scan must likely includes a port scan prior to the penetration test, which means it finds the jetty connector port.

Here is a forum thread which provides one of the initial reports of the issue.

https://forums.adobe.com/thread/2347245

I have since provided the this workaround to several customers and they have all seen the CPU spikes disappear.

Here is the edit you need to make:

1. Go to \ColdFusion2016\cfusion\lib\ 
2. Open "jetty.xml" and search for port "5500" on Connector line
3. The Host attribute shows "0.0.0.0", please replace it with 127.0.0.1. 
4. Save the changes and restart the respective ColdFusion Application service.

The 5500 port relates to a ColdFusion feature which allows you  to have the Server Monitor client communicate on its own ip\port.  It is unlikely you are using this feature so I would recommend to also uncheck this feature if you are not using it.

So if you are seeing CPU spikes, and you see them across multiple servers and across Dev and Test this could be your issue.  

Lessons Learned From Equifax

After watching some of the Congressional Hearing on the Equifax breach,  and reading articles such as https://www.engadget.com/2017/10/03/former-equifax-ceo-blames-breach-on-one-it-employee/ a few things come to mind.

One, it is a powerful reminder on how vigilant you need to be with keeping all products patched that are running on your server.

Make sure you your security team is being notified of all products security alerts, including ColdFusion.

Second, I think an area that is being missed is that the application that was breached was the Credit Score Dispute application.  That type of application should not have access to the Personal Identifiable Information, PII, database. If Equifax would have sandboxed that application better, the malicious hackers would have not been able to get to the honeypot of data.

It appears like the Credit Dispute application had a JDBC connection to the database storing the PII and thru that connection they retrieved that database schema and soon were running queries.  The congressional hearing never went into this line of questioning, I would say that this is more likely the reason the breach was so large.  If the PII database was better protected the Credit Dispute application would never been able to query the PII information.

One account I have worked with implemented an Application Tier and a Data Tier API for exactly this type of protection  If an application needed to get data it needed to send its application based credentials  for every call.  This gave them controls at the application level,  nobody could talk directly to the database, they had to go thru the Data Tier with all the security rules and tracing in place.

 If you store PII, it might be a good time to review your public sites,  make sure your DSNs are properly protected.  Think about ways to sandbox customer facing applications so that if they are breached, you still have additional barriers before they can access your PII.

ColdFusion and the Built-In Tomcat Clustering

Ever since ColdFusion 10 Tomcat has been the default J2EE Servlet engine, prior to that it was JRun.  One feature that carried forward with the Enterprise edition is the ability to Cluster running ColdFusion server instances.  This article will try and breakdown what happens when you use clustering and when you might want to use, if at all.

There are 2 basic components to what is referred to as Clustering.

1. Request Load Balancing between all the peers in the cluster
2. ability for peers to share the session scope also called in-memory session replication

Session replication is managed in the <servername>\runtime\conf\server.xml file.  The main reason for this file is to define the TCPIP multicast port so all the peers can communicate and know when they are online.

Load Balancing is accomplished using 2 Connector settings files:

1. Worker.properties file where the server instances are defined
2. URIWorkerMap.properties file is where the load balancing is defined

Note: You do not need any session replication in order to do Tomcat Connector Load Balancing, but as of now if you define a Cluster in ColdFusion you will have both of the above defined for you.

When you define a cluster in ColdFusion Enterprise the administrator will take care of editing all the above files to setup your cluster with the correct settings.

Peers can be local or remote servers.

• Session replication or sticky sessions are not required when the session storage is pointed externally to Redis on each cluster member.   Redis was added with ColdFusion 2016 as an alternate way of storing the session scope.
• If using in-memory replication you need to use J2EE Sessions for session replication or sticky sessions. All instances participating in the cluster must use J2EE sessions. Therefore, you will need to go to Server Settings ---> Memory Variables and check "Use J2EE session variables" for each cluster member.

In-memory Session Replication

As I stated above the only reason to use the multicast port setup is for in-memory session replication.  This feature works but it still takes time to get all the servers synched for every session variable change on all the cluster peers.  I would not recommend this for production servers.  The replication creates all kinds of traffic, and you never really know how fast all the sessions are synch'd.  If you have a server go down, it is not 100% that it will be a seamless hand-off, which is kind of the point to replicating in the first place.  

Sharing Sessions

There are several ways to share session or user info across servers:

1. Redis if you have ColdFusion 2016
2. Client Scope saved to Database
3. Distributed EHCache
4. Custom DB call

The easiest approach that is tried and true and does not add much overhead is to simply use the client scope, and have the Client scope defined in the ColdFusion Administrator to be saved to the database.

More advanced solutions like Redis and EHCache are out there but will likely take some time to implements and test.

Tomcat Connector Load Balancing

The load balancing that is defined is typically for instances on the same server, although it can just as easily point to remote instances.  If you have a hardware load balancer, like an F5, you probably may not need Connector based load balancing.  

The files that control the load balancing are located inside 2 files mentioned above and created when you define your connector.  

Important: If you ever upgrade your connector these settings will be removed,  so make sure to keep backups.  As of ColdFusion 11 Updater 4, CF will now archive your old connector settings.  It still clears them out, but at least they are not lost.  You will need to rebuild them as they were to get load balancing back.

HAProxy - An Open Source Software based Server Load Balancer

As an alternative to Connector load balancer you might look at using a product called HAProxy to route requests.  It has similar features as F5 but it is all software based.

If you did use HAProxy you would likely not define any Clustering in ColdFusion at all.

Website: http://www.haproxy.org/

Using the ColdFusion Admin to only Define Load Balancing

If you only want to do load balancing, I have found that you can use the Clustering page in the admin and define your cluster, then once it is saved, you can then remove or comment out the Clustering section in the <servername>\runtime\conf\server.xml file.  The result will be that you will have the Load Balancing settings completed for you.  The server will not try to find other servers, and potentially replicate sessions.

Summary

I have found that if an application is worthy of clustering, it usually will be placed under an F5 like hardware device for load balancer.  However, if for some reason you don't have access to an F5 or want to have more granular control of routing requests then setting up Load Balancing with Tomcat is a great way to do it.

I would not recommend using the multicast server peer discovery for session replication.  You can share session information in many more reliable ways.

Docker and containers are changing everything with deploying, scaling and session management, I will try and write an article building on this one for Docker soon.

Good Advice from the Apache Struts Team on keeping your Servers Patched

If you have been tracking the recent Equifax breach, it should remind all IT professionals to always have someone dedicated to keeping servers properly updated. 

I found this article and I thought the below advice from the Apache Struts team applies to any J2EE server.  It seems to apply to any publicly accessible Web or Application server product.

Please note that ColdFusion does NOT use any portion of Apache Struts.

The below advice can be found inside the complete article.here:

https://blogs.apache.org/foundation/entry/apache-struts-statement-on-equifax

Our general advice to businesses and individuals utilizing Apache Struts as well as any other open or closed source supporting library in their software products and services is as follows:

1. Understand which supporting frameworks and libraries are used in your software products and in which versions. Keep track of security announcements affecting this products and versions.

2. Establish a process to quickly roll out a security fix release of your software product once supporting frameworks or libraries needs to be updated for security reasons. Best is to think in terms of hours or a few days, not weeks or months. Most breaches we become aware of are caused by failure to update software components that are known to be vulnerable for months or even years.

3. Any complex software contains flaws. Don't build your security policy on the assumption that supporting software products are flawless, especially in terms of security vulnerabilities.

4. Establish security layers. It is good software engineering practice to have individually secured layers behind a public-facing presentation layer such as the Apache Struts framework. A breach into the presentation layer should never empower access to significant or even all back-end information resources. 

5. Establish monitoring for unusual access patterns to your public Web resources. Nowadays there are a lot of open source and commercial products available to detect such patterns and give alerts. We recommend such monitoring as good operations practice for business critical Web-based services.

Once followed, these recommendations help to prevent breaches such as unfortunately experienced by Equifax.

For the Apache Struts Project Management Committee,

René Gielen
Vice President, Apache Struts 

Building Dynamic Documents and Letters for Printing and Archiving with ColdFusion

I was recently on a LiveCycle project that involved created financial documents which were of legal significance.  Every sentence needed legal approval as each state would have different legal requirements on many different conditions.

The process worked well, but I kept thinking if the same process could be done using ColdFusion as well.  The below breaks down the workflow using LiveCycle and how I can see it done with ColdFusion.

In the end the result of the process was a PDF that was then printed and mailed.

LiveCycle Dynamic Letter Process ending in a PDF

1. Business Analyst creates the letter in Word, using colors and tracking comments to describe all the conditions.
2. The letter is sent to Legal for approval, once approved move to step 3.
3. The letter is then created in LC Designer or AEM Designer, using an LC Designer developer
1. The LC PDF uses embedded Javascript to handle the conditions based on XML values passed into the document.
2. Conditions maybe things like state and different program types as well as numeric values.
4. Another team of developers would create the server side scripts to create the input xml used in the LiveCycle Process
5. Once the letter was completed it is placed in the LiveCycle Repositiry
6. To create the letter a REST call would be made to LC Process Endpoint, passing in the XML created in step 4 along with what template needs to be created, and the process produces a PDF 
7. The letter was then returned for printing and archiving.

So how could we do this with ColdFusion assuming the keeping the process close to the above.  It is likely you will need to adapt to your needs.

This could all be done in CF using AEM Designer.

1. Business Analyst creates the letter in Word, using colors and tracking comments to describe all the conditions.
2. The letter is sent to Legal for approval, once approved move to step 3.
3. The letter is then created in LC Designer or AEM Designer, using an LC Designer developer
1. The LC PDF uses embedded Javascript to handle the conditions based on XML values passed into the document.
2. Conditions maybe things like state and different program types as well as numeric values.
4. A CF developer creates the script to create the xml to be included when the pdf is requested.
5. Once complete the letter would be placed into a resource folder.
6. To create the letter a REST call, or any cf call, would be requested, which would cfinclude the script in generate the XMLValues.
7. The letter was then returned for printing and archiving.

The CF code to replace LC and step 6 wold look similar to the below.

I found that you need to save the LiveCycle PDF inside AEM Designer as STATIC as opposed to Dynamic in order for cfpdfform to be able to work with the PDF.

In the below example I am reading straight from the LiveCycle Repository.

<cfif isdefined("form.generatepdf")>

    <cfpdfform action="populate"

        source="#rootdir#\Resources\XDP\#form.templatefilename#"

        XMLdata="#rootdir#\Resources\XML\#form.xmlfilename#"

        destination="#rootdir#\letters\#form.templatefilename#"  overwrite="true"

    />

File: #rootdire#\letters\#form.templatefilename# was generated!

Right now you can download AEM Designer using the below link:

AEM Designer can be downloaded here:

https://forums.adobe.com/external-link.jspa?url=https%3A%2F%2Fwwwimages2.adobe.com%2Fcontent%2Fdam%2FAdobe%2Fen%2Fdevnet%2Flivecycle%2Ftools%2Faemforms_designer_6_1_0_wwe_win.zip

Adobe seems to be posting the license number in this blog:

https://forums.adobe.com/thread/1192100

I'd recommend talking to Adobe on what their intentions are with licensing.

Summary of Using ColdFusion to Create Dynamic Financial Documents and Letters

Banks and Financial institutions invest heavily with enterprise solutions for creating dynamic documents and letters. Perhaps you have a scenario where this use case could apply.

The unique aspects with this use case are with the planning, legal approvals, and having the PDF use Javascript to manage all the conditions of each PDF. 

I think a high percentage of LiveCycle Forms processes can fit into a use case described above.  

Output service
You still may need to purchase LiveCycle Output for flattening and archival depending on your needs.  It all depends on your archive procedures.

http://help.adobe.com/en_US/livecycle/11.0/Services/WS92d06802c76abadb76c48dfe12dbeb3e281-7fda.2.html

Troubleshooting ColdFusion Server Outages with Unresponsive Server Alert

When I am working on a ColdFusion server issue that can't be easily identified I will define an Unresponsive Server Alert which also creates a stacktrace and is then emailed.

1. Open your ColdFusion Administrator
2. Open the Server Monitor
3. Go to the Alerts Tab
4. Select the Alert Configuration
5. Define the Alert as shown below.

Use 1 for server hung count as we are trying to capture the issue as soon as possible.  I will usually set it to 30 seconds but your time can vary based on the issue.

Important Note: In order to actually get a stack trace you need to have Monitoring turned on.
You do not need Profiling or Memory Tracking turned on.  In fact you should never turn on Memory Tracking in production.

Your top bar should look like the below.

Once you start to receive the stack traces you can start to identify patterns and hopefully get the issue solved.