Comments on: PowerExchange Throughput

By: Shane

Shane — Sun, 21 Mar 2010 01:24:31 +0000

My memory is a little fuzzy at this point, because it’s been well over three years since I’ve worked with Informatica. If I recall, it was only opening one session in Oracle and issuing one query per source. It would round-robin between the different Informatica sources. I don’t even recall what the Oracle syntax was, but it was basically “SELECT * FROM WHERE table_name = ‘table1′”, and so one for each table.

I do recall a specific test where the throughput under this one session set-up was much much faster.

Sorry I can’t be more specific, but I don’t have anything available that would help me replicate a test to demonstrate.

By: rohit

rohit — Sat, 20 Mar 2010 15:34:40 +0000

Hi,
How would having all sources in the same mapping help.All your sources will open multiple log miner oracle sessions though they are all configured in the same informatica session.Is my understanding correct?

Thanks.

By: Shane

Shane — Wed, 28 Jun 2006 19:28:56 +0000

Trevor,

Our source instance is about 250GB. There are four 100MB redo logs. During these batch cycles, we’re doing log switches — is “log switch” the correct term? — every 30 seconds. When we were testing throughput, we discussed up-ing the size of the redo to get faster reading. This was suggested by the DBA, not by us on the PowerExchange side. We never really tested this to see if it had any effect. Have you done any testing with this?

These numbers are from our production instance. We don’t have the sort of control over source transactions we would in a test instance. I measure throughput by looking at the time stamp of the target transaction: SYSDATE. The pattern tells you when there is a batch commit. You’ll see a sudden spike, 600 rows one second, 550 the next, 550 again … and then it will drop to single digits again. So, yes we are seeing latency, but it’s certainly tolerable for us.

Remember that PowerExchange is replicating data asynchronously. If sub-second latency is your goal, you might want to check out another product. We’re very happy to capture source data without impacting performance in our front-end applications.

It’s difficult to test latency in this case because the timestamps of these source transactions (dtl_capxtimestamp) are not the timestamp of the commit but of the DML action in the batch. So we’ll see a spike with source timestamps ranging over the past several hours.

At first glance, this told us we had a latency problem. Further clues — talking to application developers and seeing other transactions on the same table come through during that period — told us that this was the behavior of batch commits. Logminer “releases” the data on commit, not when the action occurs.

I’m interested in hearing how you set up your tests. We spent a LOT of time figuring out how to set up good tests, and didn’t have many peers to talk to. I’m glad to hear from you.

By: Trevor Tian

Trevor Tian — Wed, 28 Jun 2006 16:12:19 +0000

Hi,

I have some questions about your source instance, what about the size of redo log file, how many redo groups you have?

regarding to throughput, I’d like to know how you difined the duration, like start time and end time. for instance, 500 rows per second, it meant you made 500 rows changed in source instance and committed it, then data was updated in target after 1 second, right?

Thanks,
Trevor