06 Nov, 2013
1 commit
-
… to diagnose the causes of these events Functional tests found a ConcurrentModificationException with r9753 (see at the end of the commit message): The code was removing elements from a Collection while it was iterating on the exact same Collection. ReplicationBroker.java: In inner class LocalEvaluation: - Renamed "filteredRSs" to "accepted". - Added getAcceptedRSInfos() to allow iterating on them without triggering ConcurrentModificationException. In rejectAll*(), called getAcceptedRSInfos(). In getCSN(), renamed parameters. Fixed ConcurrentModificationException. ComputeBestServerTest.java: Added a test2ServersUpToDateAnd1EvenMoreUpToDate() to exhibit the ConcurrentModificationException. Extracted method newServerState() and used it in all tests. In all test methods but one, do not update the ServerState with unused CSNs because they make the tests harder to read. Did not change test3Servers() because I believe it is good to keep one test testing the unneded CSNs do not impact the test results. java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) at java.util.HashMap$ValueIterator.next(HashMap.java:822) at org.opends.server.replication.service.ReplicationBroker.rejectAllWithRSOnDifferentVMThanDS(ReplicationBroker.java:1908) at org.opends.server.replication.service.ReplicationBroker.filterServersOnSameHost(ReplicationBroker.java:1878) at org.opends.server.replication.service.ReplicationBroker.computeBestReplicationServer(ReplicationBroker.java:1600) at org.opends.server.replication.service.ReplicationBroker.connectAsDataServer(ReplicationBroker.java:820) at org.opends.server.replication.service.ReplicationBroker.connect(ReplicationBroker.java:691) at org.opends.server.replication.service.ReplicationBroker.start(ReplicationBroker.java:221) at org.opends.server.replication.service.ReplicationDomain.enableService(ReplicationDomain.java:3024) at org.opends.server.replication.service.ReplicationDomain.changeConfig(ReplicationDomain.java:3054) at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:4089) at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:96) at org.opends.server.admin.server.ServerManagedObjectChangeListenerAdaptor.applyConfigurationChange(ServerManagedObjectChangeListenerAdaptor.java:74) at org.opends.server.admin.server.ConfigChangeListenerAdaptor.applyConfigurationChange(ConfigChangeListenerAdaptor.java:342) at org.opends.server.extensions.ConfigFileHandler.replaceEntry(ConfigFileHandler.java:1610) at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processModify(LocalBackendModifyOperation.java:599) at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processLocalModify(LocalBackendModifyOperation.java:299) at org.opends.server.workflowelement.localbackend.LocalBackendWorkflowElement.execute(LocalBackendWorkflowElement.java:690) at org.opends.server.core.WorkflowImpl.execute(WorkflowImpl.java:197) at org.opends.server.core.WorkflowTopologyNode.execute(WorkflowTopologyNode.java:100) at org.opends.server.core.ModifyOperationBasis.run(ModifyOperationBasis.java:451) at org.opends.server.core.SynchronousStrategy.enqueueRequest(SynchronousStrategy.java:49) at org.opends.server.protocols.ldap.LDAPClientConnection.addOperationInProgress(LDAPClientConnection.java:1276) at org.opends.server.protocols.ldap.LDAPClientConnection.processModifyRequest(LDAPClientConnection.java:2268) at org.opends.server.protocols.ldap.LDAPClientConnection.processLDAPMessage(LDAPClientConnection.java:1745) at org.opends.server.protocols.ldap.LDAPRequestHandler.run(LDAPRequestHandler.java:194) git-svn-id: https://svn.forgerock.org/opendj/trunk@9784 41b1ffd8-f28e-4786-ab96-9950f0a78031
04 Nov, 2013
2 commits
-
…d (RSUpdater) can spin This change is linked to the misbehaving RSUpdater thread which (wrongly): - could be started multiple times - would not shutdown willingly when the server is shutting down - would try to look for replay operations in the future LDAPReplicationDomain.java: In buildAndPublishMissingChanges(), only exit the loop when the currentStartCSN is newer than now (instead of the incorrect currentStartCSN + 10s). HistoricalCsnOrderingTest.java: Code cleanup. git-svn-id: https://svn.forgerock.org/opendj/trunk@9767 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
…d (RSUpdater) can spin This change is linked to the misbehaving RSUpdater thread which (wrongly): - could be started multiple times - would not shutdown willingly when the server is shutting down - would try to look for replay operations in the future LDAPReplicationDomain.java: Added AtomicReference for RSUpdater thread. Used DirectoryThread.isShutdownInitiated() for ServerStateFlush and RSUpdater threads. In RSUpdater, added shutdown field to pass it down to buildAndPublishMissingChanges() + overrode initiateShutdown() to set the shutdown field to true. In shutdown(), called initiateShutdown() for RSUpdater and RSUpdater threads. In sessionInitiated(), only start the RSUpdaterThread if it does not already exist. In buildAndPublishMissingChanges(), added early exits in case of shutdown. Added now() method. HistoricalCsnOrderingTest.java: Consequence of the change to LDAPReplicationDomain. git-svn-id: https://svn.forgerock.org/opendj/trunk@9764 41b1ffd8-f28e-4786-ab96-9950f0a78031
29 Oct, 2013
2 commits
-
AssuredReplicationServerTest.java Fixed lost configuration lost in r9746. FakeReplicationDomain.java, ReplicationDomainTest.java: Added back configuration accidentally removed in r9746. git-svn-id: https://svn.forgerock.org/opendj/trunk@9759 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
In createReplicationBroker(), added back configuration accidentally removed in r9746 git-svn-id: https://svn.forgerock.org/opendj/trunk@9758 41b1ffd8-f28e-4786-ab96-9950f0a78031
28 Oct, 2013
2 commits
-
Used Assertions to have more descriptive assertion failures. git-svn-id: https://svn.forgerock.org/opendj/trunk@9754 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
…r in order to diagnose the causes of these events This change records how each replication server is evaluated by computeBestReplicationServer(). It also logs why a DS switches from one RS to another RS. ReplicationBroker.java: Added static classes RSEvaluations and LocalEvaluation. Moved keepBest() method to RSEvaluations. In filterServersWithAllLocalDSChanges(), reworked the code to be more understandable + Added method getCSN() and rejectAllWithRSIsLaterThanBestRS(), rejectAllWithRSOnDifferentVMThanDS(). In computeBestServerWhenConnected(), extracted method isServerOverloadingRS(). replication.properties: Added tons of messages. ComputeBestServerTest.java: Moved this class to package org.opends.server.replication.service to match the package of the tested code. Consequence of the changes to ReplicationBroker. Added methods containsOnly(), getDescription(), getEval1() and getEval2(). Reformatted the code a bit. Removed @throws from javadocs. git-svn-id: https://svn.forgerock.org/opendj/trunk@9753 41b1ffd8-f28e-4786-ab96-9950f0a78031
23 Oct, 2013
1 commit
-
Replaced instance fields with directly storing and using the ReplicationDomainCfg object. ReplicationBroker.java: Replaced instance fields with directly storing and using the ReplicationDomainCfg object. Removed start(Set<String>), setChangeTimeHeartbeatInterval() (now useless), and setGroupId() (never used). Added getBaseDN(), getReplicationServerUrls(), getGroupId(). In several methods, used Map.entrySet() instead of Map.keySet() with Map.get(Object) + renamed replicationServerInfo to rsInfo. replication.properties: Removed now unused message. ReplicationDomain.java: Replaced instance fields with directly storing and using the ReplicationDomainCfg object. Code cleanups: - Removed useless use of "this." for method calls. - In processErrorMsg() and processUpdateDone(), collapsed if statements - Increased vertical density - Extracted method getGenId() - In waitForAckIfAssuredEnabled(), used early exit. DomainFakeCfg.java: Comments => javadocs baseDn => baseDN Removed several unused methods. Added setWindowSize() + instance member. *Test*.java Used the DomainFakeCfg everywhere + SortedSet instead of Set. git-svn-id: https://svn.forgerock.org/opendj/trunk@9746 41b1ffd8-f28e-4786-ab96-9950f0a78031
17 Oct, 2013
2 commits
-
…e directory server's broker was not connected to a replication domain, hence it was impossible to add the changes to the changelogDB and then find them when querying the changelogDB. GenerationIdTest.java: Renamed assertConnectedToReplicationDomain() to waitConnectionToReplicationDomain(). LDAPReplicationDomain.java: Removed uses of StringBuilder, relied on *Operation.toString() to do it for us. ReplicationBroker.java: Improved toString(). ReplicationTestCase.java: Removed a wrong comment. Used Assertions.assertThat(). ReplicationServerTest.java: Removed a useless call to DirectoryServer.getSynchronizationProviders(). git-svn-id: https://svn.forgerock.org/opendj/trunk@9736 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
… that it is failing because the server is not connected to a replication domain, hence it is impossible to add the changes to the changelog. GenerationIdTest.java: Added assertConnectedToReplicationDomain() + called it. Fixed a few logged strings. git-svn-id: https://svn.forgerock.org/opendj/trunk@9735 41b1ffd8-f28e-4786-ab96-9950f0a78031
16 Oct, 2013
1 commit
-
Used StaticUtils.stackTraceToSingleLineString(e)) for logging on one line only. ReplicationBackend.java: Renamed all *containers to *domains. Reworked findExportContainers() and findSearchContainers() into only one method: selectReplicationDomains(). ListenerThread.java: Fixed formatting. ReplicationServerLoadBalancingTest.java: Code cleanup. git-svn-id: https://svn.forgerock.org/opendj/trunk@9729 41b1ffd8-f28e-4786-ab96-9950f0a78031
15 Oct, 2013
3 commits
-
More code cleanup. git-svn-id: https://svn.forgerock.org/opendj/trunk@9728 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Code cleanup. git-svn-id: https://svn.forgerock.org/opendj/trunk@9727 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
UpdateOperationTest.java: Added consumeAllMessages(ReplicationBroker). In csnGeneratorAdjust(), called consumeAllMessages(). Extracted local variables serverId. Fixed formatting. Increased vertical density. Removed a useless call to Thread.sleep(). Removed useless comments. ReplicationBroker.java: Used local variables "localSession" to avoid concurrency issues when setSession(null) is called by another thread. *.java: Implemented toString(). git-svn-id: https://svn.forgerock.org/opendj/trunk@9725 41b1ffd8-f28e-4786-ab96-9950f0a78031
14 Oct, 2013
1 commit
-
Tried to simplify the code + separated similar looking test data between the different tests to help debug the (not so) random test failure. git-svn-id: https://svn.forgerock.org/opendj/trunk@9724 41b1ffd8-f28e-4786-ab96-9950f0a78031
11 Oct, 2013
2 commits
-
ReplicationTestCase.java: In executeTask() and waitTaskState() added maxWaitTimeInMillis parameter. HistoricalTest.java: In testRecurringPurgeIn1Run(), added a sleep before configuring conflicts purge delay + waited purge for a maximum of 120 seconds (was approximatively 20 seconds before): as long as task max duration. InitOnLineTest.java, ReplicationServerTest.java: Consequence of the change to ReplicationTestCase. GenerationIdTest.java: Consequence of the change to ReplicationTestCase. Extracted method createSetGenerationIdTask(). git-svn-id: https://svn.forgerock.org/opendj/trunk@9719 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
In computeInitialServerStatus(), used early exits. In computeBestServerForWeight(), extracted methods computeBestServerWhenNotConnected() and computeBestServerWhenConnected(). Changed replicationServerUrls from Collection<String> to Set<String>. Removed useless field initialization to null. Renamed _publish() to publish() + reduced local variables scope. In receive(), renamed local variable replicationServerID to previousRsServerID and used this one more rather than the field. In changeConfig(), used Set.equals(). Changed getReplicationMonitor() to getReplicationMonitorInstanceName(). ReplicationDomain.java: Consequence of the change to ReplicationBroker.getReplicationMonitor(), inlined getReplicationMonitorInstanceName(). Consequence of the change to ReplicationBroker.replicationServerUrls. TestCaseUtils.java: Moved newSet(), newSortedSet(), newList() here from test classes. *Test.java: Consequence of the changes to ReplicationBroker.replicationServerUrls. git-svn-id: https://svn.forgerock.org/opendj/trunk@9712 41b1ffd8-f28e-4786-ab96-9950f0a78031
10 Oct, 2013
8 commits
-
git-svn-id: https://svn.forgerock.org/opendj/trunk@9706 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Changes after review from Matthew Swift. Renamed CNIndexRecord to ChangeNumberIndexRecord to be consistent with ChangeNumberIndexDB name. git-svn-id: https://svn.forgerock.org/opendj/trunk@9703 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Removed ChangeNumberIndexDB.getRecord(long). ECLServerHandler.java: Replaced calls to ChangeNumberIndexDB.getRecord() by calls to ChangeNumberIndexDB.getCursorFrom(). Inlined all the crossDomainStartState local variables. Added getCursorFrom() to preserve old behaviour from JEChangeNumberIndexDBCursor ctor. ChangeNumberIndexDB.java: Removed getRecord(). JEChangeNumberIndexDB.java: Removed getReadCursor(), getRecord() + some surprising and never used code like lock field, hasLock(), lock() and release(). JEChangeNumberIndexDBCursor.java: In ctor, do not throw ChangelogException if the changeNumber cannot be found. JEChangeNumberIndexDBTest.java: Used JEChangeNumberIndexDB.getCursorFrom() instead of JEChangeNumberIndexDB.getReadCursor(). Renamed a few variables. git-svn-id: https://svn.forgerock.org/opendj/trunk@9702 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Added tests on the CSNs returned by changelog and on their order too. Removed stupid for loops. Extracted methods readCookie() and commonAssert(). Renamed: - getCookie() to assertContainsAndReadCookie() - assertEntries() to assertFourEntries() - assertDnEquals() to assertDNEquals() Added getCSNs(). git-svn-id: https://svn.forgerock.org/opendj/trunk@9701 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
git-svn-id: https://svn.forgerock.org/opendj/trunk@9700 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Changes after review from Matthew Swift. Consolidated ChangeNumberIndexDBCursor and ReplicaDBCursor into a single interface named DBCursor. DBCursor.java: ADDED It does not implement Comparable like ReplicaDBCursor. Instead a Comparator has been extracted to JEChangelogDB.CrossReplicaDBCursor . ChangeNumberIndexDBCursor.java, ReplicaDBCursor.java: DELETED - replaced by DBCursor *.java: Used DBCursor instead of removed cursor interfaces. git-svn-id: https://svn.forgerock.org/opendj/trunk@9699 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Here is the failure scenario for this test (with one CPU): t1 (test thread) t2 (CNIndexDB purge thread) ---------------- --------------------------- | | | v | JEChangeNumberIndexDB | run() | trim() | clear(null) <- entered, but not completed v ChangeNumberIndexDB setPurgeDelay(0); addRecord(1) addRecord(2) addRecord(3) | | | +--> executes the clear() v getFirstRecord() <- BOOM NullPointerException! Ensured the CNIndexDB is not trimmed until we explicitly test it: to do this, ensured the thread is only started when needed by the test. Also took the occasion to ensure a faster shutdown for the CNIndexDB trimming thread. JEChangeNumberIndexDB.java: Ensured we use volatile and AtomicBoolean. Renamed thread field to trimmingThread. Extracted method startTrimmingThread(). In trim(), added an AtomicBoolean parameter. Added methods clear(DN, AtomicBoolean shutdown) and mustShutdown() to ensure faster shutdown. JEChangelogDB.java: Called JEChangeNumberIndexDB.startTrimmingThread() after creating the JEChangeNumberIndexDB instance. JEChangeNumberIndexDBTest.java: Extracted constants. In testTrim(), called JEChangeNumberIndexDB.startTrimmingThread(). In testClear(), removed git-svn-id: https://svn.forgerock.org/opendj/trunk@9698 41b1ffd8-f28e-4786-ab96-9950f0a78031 -
Code cleanup. Used InternalClientConnection.process*() methods. git-svn-id: https://svn.forgerock.org/opendj/trunk@9697 41b1ffd8-f28e-4786-ab96-9950f0a78031
09 Oct, 2013
3 commits
-
More code cleanups. git-svn-id: https://svn.forgerock.org/opendj/trunk@9694 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
ReplicationServerDomain.java: Renamed getStartState() to getOldestState(). ServerState.java: In duplicateOnlyOlderThan(), changed parameter from CSN to long. ReplicationServer.java: Consequence of the change to ServerState and ReplicationServerDomain. In getECLChangeNumberLimits(), brought the database empty code to the top of the method: to get something more readable. ECLServerHandler.java: Consequence of the change to ServerState and ReplicationServerDomain. Changed a few names / comments. JEChangeNumberIndexDB.java Consequence of the change to ReplicationServerDomain. ExternalChangeLogTest.java Consequence of the change to ReplicationServerDomain. Code cleanup: - removed gblCSN instance field - inlined sleep() - used connection.process*() + inlined runModifyOperation() and runDeleteOperation(). - renamed getReplicationDomainStartState() to getDomainOldestState() + added better asserts git-svn-id: https://svn.forgerock.org/opendj/trunk@9691 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
JEChangeNumberIndexDBTest.java: Fixed the build. ChangeNumberIndexDB.java, JEChangeNumberIndexDB.java: Improved the javadocs. Made instance variables volatile. *.java: First => Oldest Last => Newest git-svn-id: https://svn.forgerock.org/opendj/trunk@9690 41b1ffd8-f28e-4786-ab96-9950f0a78031
08 Oct, 2013
1 commit
-
Big code cleanup to remove as much useless variables and useless code as possible to only leave the important parts of the tests. There's more to do but I am tired now, I'll see later for the rest. git-svn-id: https://svn.forgerock.org/opendj/trunk@9684 41b1ffd8-f28e-4786-ab96-9950f0a78031
07 Oct, 2013
2 commits
-
* rename CSN comparison methods to make them easier to understand. git-svn-id: https://svn.forgerock.org/opendj/trunk@9672 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Fixing this test: the main problem was in classCleanUp() that was trying to remove non leaves entries, which ended up in a failure. git-svn-id: https://svn.forgerock.org/opendj/trunk@9669 41b1ffd8-f28e-4786-ab96-9950f0a78031
05 Oct, 2013
2 commits
-
Fixing this test: the main problem was in afterTest() that was trying to remove non leaves entries, which ended up in a failure. Renamed all the "changelog" to "replServer". Extracted a few methods. Removed useless parameters from other methods. Removed useless methods. Removed unneeded try / catch / log. Made better use of assertions and assertj. git-svn-id: https://svn.forgerock.org/opendj/trunk@9668 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Fixed problem introduced in tests at r9648. HeartBeatMsg.java: Added toString(). git-svn-id: https://svn.forgerock.org/opendj/trunk@9666 41b1ffd8-f28e-4786-ab96-9950f0a78031
04 Oct, 2013
2 commits
-
… it helps with CI that runs on single CPU machine. git-svn-id: https://svn.forgerock.org/opendj/trunk@9665 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Renamed some classes in package src.server.org.opends.server.replication.server.changelog.je: - DbHandler.java => JEReplicaDB.java (and tests) - DraftCNDbHandler.java => JEChangeNumberIndexDB.java (and tests) - DraftCNDbIterator.java => JEChangeNumberIndexDBCursor.java *.java: Consequence of the renames. Changed methods/variables/parameter names and comments to reflect the new names. In particular, bye bye to DraftCNDb, DbHandler and welcome to CNIndexDB and ReplicaDB. git-svn-id: https://svn.forgerock.org/opendj/trunk@9662 41b1ffd8-f28e-4786-ab96-9950f0a78031
03 Oct, 2013
4 commits
-
…fore proceeding. Adding a wait should solve the problem. ReplicationTestCase.java Added executeTask(). Used simpler InternalConnection.process*() APIs. HistoricalTest.java: Used ReplicationTestCase.executeTask(). Inlined local variables. Extracted method getEntryValue(). Changed signature of publishModify(). GenerationIdTest.java: Used ReplicationTestCase.executeTask(). Renamed assertNoMessageReceivedBadGenId() to assertNoMessageReceived(). ReplicationServerTest.java Used ReplicationTestCase.executeTask(). TestCaseUtils.java: Used simpler InternalConnection.process*() APIs. UpgradeTestCase.java Renamed isOutputContainsExpectedMessage() to assertContainsMessage(). *Test.java: Used assertj. Let the exceptions go up. git-svn-id: https://svn.forgerock.org/opendj/trunk@9650 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
Change entryList and configEntryList from LinkedList to Set and renamed them to entriesToCleanup and configEntriesToCleanup. Extracted method connect() and waitForSpecificMsg() + Changed the return type of the existing waitForSpecificMsg() methods. In configureReplication(), added 2 String parameters + extracted method addSynchroServerEntry() and addConfigEntry(). Simplified code a lot. *Test[Case].java: Consequence of the changes to ReplicationTestCase. Factorized code with ReplicationTestCase. Various cleanups: - used assertj - used StaticUtils.close() - made fields private - used foreach - used assertEquals() instead of assertTrue() - removed try/catch/fail test anti pattern. - let exceptions go up the stack ProtocolWindowTest.java: Extracted method searchNbMonitorEntries(). git-svn-id: https://svn.forgerock.org/opendj/trunk@9648 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
This code is never used + UpdateMsg already implements Comparable. git-svn-id: https://svn.forgerock.org/opendj/trunk@9647 41b1ffd8-f28e-4786-ab96-9950f0a78031
-
In ExternalChangeLogTest.ECLReplicationServerFullTest(), debugging somewhat allows to reproduce the problem. The sequence of events is... interesting. In ECLAfterChangelogTrim(), the last published DeleteMsg might not even have even been received on the other side when the test ends (i.e. ServerReader.run() might never have called/completed calling Session.receive()), hence clearing the changelogDB cannot clear this yet to be received message! Looking closely at this test, I cannot see any reason to publish this DeleteMsg, so I removed this code. I also took the opportunity to get rid of what look like a useless 1s sleep. ExternalChangeLogTest.java: In ECLReplicationServerFullTest(), removed sending a useless DeleteMsg that changes nothing to the test result + removed a useless 1s sleep (Yay!) + removed the call to clearChangelogDB() in the finally block (already called by a @AfterTest) git-svn-id: https://svn.forgerock.org/opendj/trunk@9646 41b1ffd8-f28e-4786-ab96-9950f0a78031
02 Oct, 2013
1 commit
-
In ExternalChangeLogTest.ECLReplicationServerFullTest(), it looks like the server does not have the time to trim the last "changetype: delete" from the previous tests method, so let's force the test to clear the changelogDB before it finishes. ExternalChangeLogTest.java: In ECLReplicationServerFullTest(), called clearChangelogDB() in the finally block to force cleaning test data. git-svn-id: https://svn.forgerock.org/opendj/trunk@9645 41b1ffd8-f28e-4786-ab96-9950f0a78031