Doug Grove created JBMESSAGING-1931:
---------------------------------------
Summary: Message not delivered to Topic afer node rejions cluster
Key: JBMESSAGING-1931
URL: https://issues.jboss.org/browse/JBMESSAGING-1...
Project: JBoss Messaging
Issue Type: Bug
Components: JMS Clustering
Affects Versions: 1.4.8.SP5
Environment: JBoss EAP 5.1.2
Reporter: Doug Grove
After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931:------------------------------------ Attachment: node2_logs.tar.gz node1_logs.tar.gz > Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Attachments: node1_logs.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931: ------------------------------------ Description: After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Note that this configuration has the cluster health mbean installed. was:After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes.> Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Attachments: node1_logs.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931: ------------------------------------ Description: After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. Note that this configuration has the cluster health mbean installed. was: After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Note that this configuration has the cluster health mbean installed.> Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Attachments: node1_logs.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Yong Hao Gao reassigned JBMESSAGING-1931: ----------------------------------------- Assignee: Yong Hao Gao > Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1_logs.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Yong Hao Gao commented on JBMESSAGING-1931: ------------------------------------------- hi Doug, Can you upload the messaging configuration files of the two nodes? Thanks, Howard> Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1_logs.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: node1-config.tar.gz > Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: node2-config.tar.gz > Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove commented on JBMESSAGING-1931:----------------------------------------- There you go. Please let me know if you need anything else. Doug > Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Yong Hao Gao commented on JBMESSAGING-1931: ------------------------------------------- Thanks Doug. It seems that when you disconnect node 2 in the test, the DB is also disconnected from node 1. Node 1 is watching the timestamp of node 2 by periodically querying the JBM_CLUSTER table in the DB. If node 1 found it cannot see the timestamp of node 2 be updated for sometime (basically it is about two times the value of NodeStateRefreshInterval, i.e. 2*20 = 40 sec in this test), it will think that node 2 is really dead and perform failover for it. In the test you waited for 1 min, so the node 1 (as shown in the log) eventually performed failover for node 2 before you reconnected it, and the cluster will not be re-formed correctly. I'd suggest you increase NodeStateRefreshInterval to more than 30 sec to make sure failover won't happen before your waiting time, and redo the test and see how it goes. Howard> Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove commented on JBMESSAGING-1931: ----------------------------------------- I can confirm that if I set NodeStateRefreshInterval to something like 120000, and the node failover does not occur, then messages are delivered to both nodes. I'll pass this along to the customer as a temporary work-around. I would still like to pursue this issue. We should not have a situation where messages are not delivered to all nodes. Thanks much, Doug> Message not delivered to Topic afer node rejions cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Summary: Message not delivered to Topic afer node rejoins cluster (was: Message not delivered to Topic afer node rejions cluster) > Message not delivered to Topic afer node rejoins cluster > -------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Summary: Message not delivered to Topic after node rejoins cluster (was: Message not delivered to Topic afer node rejoins cluster) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931:------------------------------------ Attachment: node2-server.log.tar.gz node1-server.log.tar.gz > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931:------------------------------------ Attachment: node1-server.log.tar.gz node2-server.log.tar.gz > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931:------------------------------------ Attachment: node2-server.log.tar.gz node1-server.log.tar.gz > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931:------------------------------------ Attachment: node2-server.log.tar.gz node1-server.log.tar.gz > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-config.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node1-config.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node1-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node1-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node1-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node1-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1-server.log.tar.gz, node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node1-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: node1_logs.tar.gz, node2-config.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2-server.log.tar.gz, node2_logs.tar.gz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node1_logs.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node2-config.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node2-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node2-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node2-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node2-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node2-server.log.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: (was: node2_logs.tar.gz) > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ] Doug Grove updated JBMESSAGING-1931:------------------------------------ Attachment: jboss-messaging.jar jboss-messaging-client.jar > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove updated JBMESSAGING-1931: ------------------------------------ Attachment: JBPAPP-9758-signed.tgz > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar, JBPAPP-9758-signed.tgz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.
[ https://issues.jboss.org/browse/JBMESSAGING-1... ]Doug Grove commented on JBMESSAGING-1931: ----------------------------------------- The root cause for this issue was determined to be that messages were not being delivered to the MDB containers after JBM restarted on the node. This was addressed by getting a list of mbeans on the server and calling the stopDelivery and startDelivery methods. These are implemented by org.jboss.ejb3.mdb.MessagingDelegateWrapper. > Message not delivered to Topic after node rejoins cluster > --------------------------------------------------------- > > Key: JBMESSAGING-1931 > URL: https://issues.jboss.org/browse/JBMESSAGING-1... > Project: JBoss Messaging > Issue Type: Bug > Components: JMS Clustering > Affects Versions: 1.4.8.SP5 > Environment: JBoss EAP 5.1.2 > Reporter: Doug Grove > Assignee: Yong Hao Gao > Attachments: jboss-messaging-client.jar, jboss-messaging.jar, JBPAPP-9758-signed.tgz > > > After a node leaves the cluster and then rejoins the cluster, the node does not receive messages posted to clustered topics that are delivered to other nodes. > Log files are attached. Node 2 was the node taken out of the cluster. After Node 2 was allowed to rejoin the cluster, a message was sent to Node 1. It was not seen by Node 2. A messages was then sent to Node 2. It was seen by Node 1. Search for "Hello World" in the server.log. > Note that this configuration has the cluster health mbean installed.