Using the connection_rate parameter to stop DoS attacks

I recently posted on the oracle-l mailing list about how to stop  denial of serice attack. My message is below

We had an application that repeatedly connects to the database via java connection pool fail because the account had become locked. The application kept on trying, the database did not allow the connection and we ended up with thousands of ‘dead’ processes causing the unix server to hang as all memory was used up.

The obvious thing to fix in our case was some form of application logic to recognise that failed connections had been made and stop the repeated connection attempts.

However this could also be used in a denial of service attack. What steps could we take to reduce that risk. The problem as I see it is that the database has reacted correctly and there is not much more we could do at the database level. However I am always open to suggestions.

I received two responses,  both of which were valuable. Freek DHooge suggested enabling dead connection detection by using the sqlnet.expire time setting and another mail from Grzegorz Goryszewski directing me to the 11g new feature listener connection rate feature. I set up a test to use both features and here are the results.

Firstly I generated 3 scripts to generate a number of connections into the database

DOA.sh to open  sqlplus connections, DOAStart.sh to generate lots of calls of that script and DOAStop.sh to kill all the connections

#!/bin/sh

#
# Simple shell script to simpulate a DOA
#

while true
do
sqlplus -s << EOF
<a href="mailto:soe/soe@DB11G">soe/soe@DB11G</a>
<a href="mailto:soe/soe@DB11G">soe/soe@DB11G</a>
<a href="mailto:soe/soe@DB11G">soe/soe@DB11G</a>
EOF
done
#!/bin/sh
#
# DOA Controller
#

count=50

while [ $count -gt 0 ]
do
count=`expr $count - 1`
/home/oracle/DOA.sh > /dev/null &
done

ps -fu oracle | grep DOA | grep -v grep | awk '{print$2}' > /home/oracle/DOAProcesses

#!/bin/sh
#
# DOA Stop
echo Stopping DOA Processes
for i in `cat /home/oracle/DOAProcesses`
do
kill -9 $i
done

DOAProcCnt=`ps -ef | grep DOA | grep -v grep | wc -l`

echo Number processes Running are $DOAProcCnt

Firstly I should mention that I tried running with Swingbench and that just sends a blast of 50 logins in at once which is not what I was trying  to emulate.  I also locked the soe account I was using which is what the status would be in a real; world attack after 3 failed login atempts.

Enabling sqlnet.expire_time=1 in the sqlnet.ora file did not work and we still had a lot of dead connection which killed the CPU. Note my original problem was with an application grabbing memory but I need to spend more time testing that specific issue whereas this blog is more about the usage of the connection rate parameter in listener.ora which was new to me.

sqlnet.expire_time set

I then unset that paremeter and added the the rate_limit and connection_rate parameters to my listener .This allows 2 connections per second. Note the listener needs restarting, a reload will not be sufficient

LISTENER_server =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = server)  (PORT = 1525) (RATE_LIMIT=YES))
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1525))
)
)

SID_LIST_LISTENER_server =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /app/oracle/product/10.2.0.4/db_1)
(PROGRAM = extproc)
)
)

CONNECTION_RATE_LISTENER_server=2

connection_rate parameter enabled

Bingo – the server usage from glance was vastly improved. No died sqlplus processes and CPU not impacted. However whilst tailing the listener log file I could see no evidence that connections were being rejected.

This is a piece of work in progress and I thought I would post my initial findings and follow up later on with more findings.