searching best solution for error handling about btm HOT 1 CLOSED

I can't say much with so few details. By default, ActiveMQ does not send rolled back messages to the DLQ, but it redelivers them. Have you changed that default, or are you speaking about multiple rolled back redeliveries?

You usage of the word "transaction" is a bit confusing too:

after reading a message from Active MQ My transaction starts
What transaction? If you're speaking about a JTA transaction that encompasses your JMS and JDBC resources, it has to be started before you read the JMS message.
...while doing second database transaction...
Again, a confusing use of the word transaction.

Since you seem to be having architectural / design questions, nothing related to BTM itself, you should probably seek for help somewhere else.

from btm.