org.forgerock / org.forgerock.opendj

24 Apr, 2014

1 commit

23 Apr, 2014

2 commits

  • be9a267d   OPENDJ-1259 Make the Medium Consistency Point support replicas temporarily leaving the topology ... Browse Dir » 
    Preliminary work, also needed by Nicolas for OPENDJ-1177.


ByteSequenceReader.java:
Added peek() and peek(int offset).

ByteSequenceReaderTest.java:
Added a test.
Added method b() to ease reading in int -> byte conversion.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10670 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • e8735348   OPENDJ-1448 Remove JReplicaDB flushing thread and msg queue ... Browse Dir » 
    It seems unnecessary to have a msgQueue on top of JE since JE's already has a builtin cache that handles the same responsibility.
This improvement removes the JEReplicaDB.msgQueue and the associated flushing thread to save on memory and resources.

Code cleanup in JEChangeNumberIndexDB after CR-3388.


JEReplicaDB.java:
Does not implement Runnable anymore.
Removed fields msgQueue, queueSizeBytes, queueMaxBytes, thread, flushLock.
Added and used shutdown field to compensate for removing the thread field.
Removed methods collectAllPermits(), flush(), run() and stop().
In shutdown(), used AtomicBoolean.compareAndSet().

ReplicationDB.java:
Renamed addEntries(List<UpdateMsg>) to addEntry(UpdateMsg).

JEReplicaDBTest.java:
Removed now unnecessary waitChangesArePersisted().


JEChangelogDB.java:
In shutdown(), enforced threads joining + called Thread.interrupt() to ensure shutdown. This prevents message about unclosed cursors in integrated unit tests.

ReplicationServer.java:
Removed getQueueSize().


JEChangeNumberIndexDB.java:
Removed unused oldestChangeNumber.
In shutdown(), used AtomicBoolean.compareAndSet() + removed useless call to notify().

JEChangeNumberIndexDBTest.java:
Fixed javadocs.


ChangeTimeHeartbeatMsg.java:
Implemented toString().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10669 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

18 Apr, 2014

1 commit

  • fe2017f0   OPENDJ-1430 Some changes are missing from the external changelog changeNumber ... Browse Dir » 
    The bug was due to a very complex interaction between various components. Here is a scenario and explanation:
1- the change number indexer has no more records to proceed, because all cursors are exhausted, so it calls wait().
2- a new change Upd1 comes in for an exhausted cursor, medium consistency cannot move.
3- a new change Upd2 comes in for a cursor that is not already opened, medium consistency can move, so wake up the change number indexer.
3- on wake up, the change number indexer calls next(), advancing the CompositeDBCursor, which recycles the exhausted cursor, then calls next() on it, making it lose its change. CompositeDBCursor currentRecord == Upd1.
4- on the next iteration of the loop in run(), a new cursor is created, triggering the creation of a new CompositeDBCursor => Upd1 is lost. CompositeDBCursor currentRecord == Upd2.

The problem comes from two parts:
- CompositeDBCursor consumes next change from a cursor (which completely forget about this change) and stores it itself
- ChangeNumberIndexer manages recycling/creating cursors on its own and recreates CompositeDBCursor when a new cursor is created.

The fix required:
- Preventing CompositeDBCursor from consuming changes from underlying cursors until it can forget about this same change itself.
- Ensuring only ChangeNumberIndexer handle recycling the cursors it owns instead of having both CompositeDBCursor and ChangeNumberIndexer trying to do it. It is also more performant to let ChangeNumberIndexer manage its cursors.



CompositeDBCursor.java:
Added recycleExhaustedCursors field to tell the composite whether it can recycle the cursors itself or not (recycling the cursors is currently needed for persistent searches on the changelog, maybe will we be able to remove it in the future, that would simplify the code a lot).
Modified the ctor to pass in value of recycleExhaustedCursors.
Removed currentRecord and currentData fields, replaced by reading the record and field on the first entry in the cursors SortedMap.
Added state field to ensure the first call to next() does not consume the first change in the cursors SortedMap.

ChangeNumberIndexer.java:
ChangeNumberIndexer now manages alone the cursors recycling and creation and recreates the CompositeDBCursor when needed.
In run(), removed the now unneeded call to next() after the wait.
Added recycleExhaustedCursors().

JEChangelogDB.java:
Consequence of the change to CompositeDBCursor. Kept old recycling behaviour.


ChangeNumberIndexerTest.java:
Added emptyDBTwoDSsDoesNotLoseChanges() to cover the case being fixed by current commit.
Renamed test methods dropping the "Initial" when it was not adding much to the test comprehension.

CompositeDBCursorTest.java:
In newUpdateMsg(), added toString() implementation to help debug.
Removed recycleTwoElementCursorsTODOJNR().
In recycleTwoElementCursors(), changed the tests a bit to match the changes to CompositeDBCursor.


git-svn-id: https://svn.forgerock.org/opendj/trunk@10667 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

17 Apr, 2014

2 commits

  • 3fb4923f   OPENDJ-1440 On startup, change number can progress without waiting for any heart… ... Browse Dir » 
    …beat from known replicas

On startup, if a replication server knows about several replicas, then it must wait to receive some sort of alive information for each of them before being able to move the medium consistency forward.
Changes or heartbeats received after replication server started are acceptable.
Likewise, changes that would have been received before the replication server stopped are also acceptable.

This was fixed on replication server startup, by initializing the lastAliveCSN for each known replica, with the oldest possible CSN (timestamp == 0).
Then when checking if the medium consistency can move forward, if no medium consistency is set, then the lastAliveCSN for each known replica must have a timestamp != 0.


ChangeNumberIndexer.java:
In canMoveForwardMediumConsistencyPoint(), call allInitialReplicasArePastOldestPossibleCSN() if the medium consistency CSN is not set.
Added methods oldestPossibleCSN(), allInitialReplicasAreAlive().

ChangeNumberIndexerTest.java:
In emptyDBTwoInitialDSs(), slightly modified the code to test current bug.
In startCNIndexer(), added the initial ECL enabled domains as a parameter.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10666 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 8d301231   OPENDJ-1439 Change number stops progressing with cross domain replication ... Browse Dir » 
    The changeNumber progression was blocked when the mediumConsistencyCSN was from a different baseDN than the new change to add to the changeNumber index DB.
Fixed this problem by also storing the baseDN of the mediumConsistencyCSN in the ChangeNumberIndexer class.


ChangeNumberIndexer.java:
Renamed mediumConsistencyCSN to mediumConsistency and changed its type from CSN to Pair<DN, CSN>.
In tryNotify(), removed the now useless baseDN parameter (replaced with mediumConsistency field).
Improved comments.

ChangeNumberIndexerTest.java:
Renamed BASE_DN to BASE_DN1.
Added BASE_DN2 and test emptyDBTwoInitialDSsDifferentDomains().
In assertExternalChangelogContent(), did some renaming.

SequentialDBCursor.java:
Improved toString().

ChangeNumberIndexDB.java:
Removed obsolete FIXME.


git-svn-id: https://svn.forgerock.org/opendj/trunk@10665 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

14 Apr, 2014

1 commit

  • 3ff95f31   OPENDJ-1430 - Some changes are missing from the external changelog ... Browse Dir » 
    Code cleanup in ECLServerHandler:
The eligibleCSN was used when the ECLServerHandler was responsible for inserting replicaDB changes into the changeNumber index DB.
To this end, the eligibleCSN was computed to provide a kind of "medium consistency point" ("kind of" because it was badly computed).
After OPENDJ-1174, it is no longer ECLServerHandler's responsibility to inserting replicaDB changes into the changeNumber index DB, so the eligibleCSN is now useless and this commit removes it.

In addition this commit enhances the toString() methods for a better readability.



ECLServerHandler.java:
Removed eligibleCSN field + refreshEligibleCSN().
Renamed computeNextEligibleMessageForDomain() to computeNextAvailableMessage() + simplified code.
In DomainContext, removed nextNonEligibleMsg field + removed isEligible(), debugInfo() and toString(CSN).
In dumpState(), made the debug string more readable.
In buildDomainContexts(), used DomainContext new ctor + extracted method newDomainContext() to build full DomainContext objects in one go + fixed a possible ConcurrentModificationException.
In DomainContext, made some fields final + added a ctor + made toString() more readable.

ReplicationServerDomain.java:
Removed getEligibleCSN() and isServerConnected(), now unused.

ReplicationServer.java:
Removed getEligibleCSN(), now unused.

ReplicationDomainDB.java, JEChangelogDB.java, ChangeNumberIndexer.java:
Removed getDomainLastAliveCSNs(), now unused.

ExternalChangeLogTest.java
Consequence of the changes above.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10655 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

03 Apr, 2014

1 commit

  • dded3404   OPENDJ-1177 (CR-3314) Re-implement changelog purging logic ... Browse Dir » 
    JEChangelogDB.java:
In ChangelogDBPurger.run(), changed code to:
- support the absence of change number index DB. To simplify matters, code assumes that ds-cfg-compute-change-number does not change during the life time of an RS.
- purged by using a CSN made up from the purge delay rather than using the previous cookie.
- sleep 500 millis if there are no changes to purge, or sleep till the next change to purge.
- gracefully shutdown without fuss in the logs.


JEChangeNumberIndexDB.java
In purgeUpTo(), return the oldest non purged CSN rather than the previous cookie + merged two branches of the code.

ExternalChangeLogTest.java:
Consequence of the change to JEChangeNumberIndexDB.
Extracted method assertECLLimits() from ECLCompatTestLimits() + added a loop inside it to let the code persist changes asynchronously from the test thread.

JEChangeNumberIndexDBTest.java:
Renamed testTrim() to testPurge().
In newReplicationServer(), enabled ds-cfg-compute-change-number.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10617 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

02 Apr, 2014

1 commit

  • 66ac3ce0   OPENDJ-1177 (CR-3304) Re-implement changelog purging logic ... Browse Dir » 
    
After OPENDJ-1174, change number index DB is now populated eagerly (was populated lazily).
This means we can reap the benefits by changing how purging is done for changelogDB.
Previous purge process was driven by the replicaDBs: first purge replicaDBs then purge the change number index DB and it was causing lots of problems like stale CNIndexDB records for example.
New purge process is driven by the change number index DB: first purge change number index DB then purge the replicaDBs based on the oldest valid referenced record in the change number index DB.

Moved JEChangeNumberIndexDB purge thread to JEChangelogDB + made it responsible for purging the ChangeNumberIndexDB and all the ReplicaDBs.
In JEReplicaDB, thread is now only responsible for flushing, previously it was responsible for trimming and flushing which complicated the design and made it less efficient for both operations.



JEChangelogDB.java:
Added inner class ChangelogDBPurger.
Added fields  purgeDelay, cnPurger and latestTrimDate.
Extracted method startIndexer()
Removed getChangeNumberIndexDB(boolean) + associated code.
Reimplemented getDomainLatestTrimDate() and setPurgeDelay().

JEChangeNumberIndexDB.java:
No longer implements Runnable + removed run().
Removed fields trimmingThread, trimAge and replicationServer.
In ctor, removed ReplicationServer parameter.
Removed startTrimmingThread(), setPurgeDelay() clear(DN, AtomicBoolean).
Renamed clear(DN) to removeDomain(DN).
Renamed trim(AtomicBoolean) to purgeUpTo(long).
Added purgeUpToCookie(ChangeNumberIndexRecord).

JEReplicaDB.java:
Removed fields latestTrimDate and trimAge.
In run(), no longer call trim().
Removed getLatestTrimDate(), isQueueAboveLowMark(), setPurgeDelay() and getQueueSize().
Renamed trim() to purgeUpTo(CSN).
Made flush() private + reduced wait time on polling the msgQueue to speed up shutdown.
Added getNumberRecords() for unit tests.

JEReplicaDBCursor.java:
Since flushing is now eager, removed all calls to JEReplicaDB.flush().
Extracted methods closeCursor().

ReplicationServer.java:
Renamed getTrimAge() to getPurgeDelay() for consistency.

ReplicationDB.java:
Added getNumberRecords().

ChangeNumberIndexer.java, ChangeNumberIndexerTest.java:
Removed hacks due to old purging code.

ExternalChangeLogTest.java:
Called ChangelogDB.setPurgeDelay() instead of ChangeNumberIndexDB.setPurgeDelay(0).
Consequence of the changes to JEReplicaDB.
Removed method setPurgeDelayToInitialValue() that was not doing anything (JEChangeNumberIndexDB was always null).
In getCNIndexDB(), removed useless null check.

JEChangeNumberIndexDBTest.java:
Removed constants value1, value2, value3 which are invalid cookies.
Replaced them with fields previousCookie and cookies.
Added clearCookie() method.
In addRecord(), removed cookie parameter and build the cookie from the new fields.
Consequence of the changes to JEChangeNumberIndexDB.

JEReplicaDBTest.java:
In waitChangesArePersisted(), used JEReplicaDB.getNumberRecords() instead of JEReplicaDB.getQueueSize() + added parameters describing the number of expected records + the counter record window.
Replaced all calls to JEReplicaDB.flush() with calls to waitChangesArePersisted().
Consequence of the changes to JEReplicaDB.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10614 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

28 Mar, 2014

1 commit

  • 41b9b427   OPENDJ-1177 (CR-3278) Re-implement changelog purging logic ... Browse Dir » 
    In JEReplicaDB, simplified the logic that handled the internal queue that is used before actually persisting UpdateMsg changes to the underlying Berkeley JE DB.
Simplified the publisher/consumer model (msgQueue.add() / msgQueue.remove()) by relying on a LinkedBlockingQueue and a semaphore, instead of many synchronized blocks and fields that cluttered this code.



JEReplicaDB.java:
Changed msgQueue from LinkedList to LinkedBlockingQueue.
Removed fields queueMaxSize, queueLowmark, queueHimark, queueLowmarkBytes, queueHimarkBytes, queueByteSize and replaced them all with queueSizeBytes Semaphore.
Removed clearQueue() and getChanges().
Added collectAllPermits().
Added immutable CSNLimits class to remove the need for synchronizing on oldest and newest CSNs.

ReplicationDB.java:
In addEntries(), now return the total size of the persisted messages (return type was void).

JEReplicaDBTest.java:
In testTrim(), allowed the test to finish + made the code clearer.

replication.properties:
Added an error message for adding a change to the JEReplicaDB.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10601 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

23 Mar, 2014

1 commit

20 Feb, 2014

1 commit

19 Feb, 2014

1 commit

  • 6d1da379   OPENDJ-1271 (CR-3008) dsreplication pre-external-initialization task fails with STOPPED_BY_ERROR ... Browse Dir » 
    
Improved design for Replication Topology


ReplicationBroker.java + *Test.java:
Extracted the Topology class to encapsulate the dsList and replicationServerInfos fields + atomically set it via an AtomicReference + moved setLocallyConfiguredFlag(), isSameReplicationServerUrl(), computeConnectedDSs() to Topology class.
Created methods computeNewTopology() and topologyChange() to compute and set the new topology.
Removed generationID instance variable duplicated with the one from ReplicationDomain + updated ctor and setGenerationID().
Improved debugging messages.
Renamed getDsList() to getReplicaInfos() + changed return type from List<DSInfo> to Map<Integer, DSInfo>.
Renamed getRsList() to getRsInfos().
Extracted method toRSInfos().

ReplicationBrokerTest.java: ADDED
Added to test new ReplicationBroker.Topology class.


DSInfo.java:
Changed equals(Set<String>, Set<String>) to equals(Object, Object).
In toString(), hid the assured fields if assured replication is off.

RSInfo.java:
Renamed fields id and serverUrl to rsServerId and rsServerURL.
In toString(), relied on the compiler to generate the String.


TopologyMsg.java:
Renamed fields dsList and rsList to replicaInfos and rsInfos.
Renamed getDsList() to getReplicaInfos() + changed return type from List<DSInfo> to Map<Integer, DSInfo>.
Renamed getRsList() to getRsInfos().
Extracted methods readStrings() and writeStrings().
Code cleanup
Used javadocs


ReplicationDomain.java, replication*.properties:
Extracted class ECLIncludes to replace eclIncludesLock, eclIncludesByServer, eclIncludesAllServers, eclIncludesForDeletesByServer, eclIncludesForDeletesAllServers + moved setEclIncludes() implementation to this class + removed synchronized blocks from all the getters.
Added baseDN to ERR_INIT_NO_SUCCESS_START_FROM_SERVERS.
Inlined initializeRemote().
Removed unused initializeFromRemote().
Reduced methods visibility.
Renamed getReplicaList() to getReplicaInfos() + changed return type from List<DSInfo> to Map<Integer, DSInfo>.
Renamed getRsList() to getRsInfos().
In initializeRemote(), waitForRemoteEndOfInit(), isRemoteDSConnected() and getProtocolVersion(), simplified code.

*.java:
Consequence of the changes above.
Simplified code in a number of places, particularly in the tests.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10405 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

31 Jan, 2014

1 commit

30 Jan, 2014

1 commit

29 Jan, 2014

1 commit

28 Jan, 2014

3 commits

  • cefa3add   Misc improvements to replication Session error reporting. ... Browse Dir » 
    
replication*.properties:
Added exception stacktraces to NOTICE_READER_EXCEPTION_53.

ServerReader.java:
In run(), logged the stacktrace when calling logError(). Removed redundant call to logException() which logged to debug logger.


Session.java:
Extracted method read().


StaticUtils.java:
In stackTraceToSingleLineString(), when this is not a debug build, added the exception type at the start of the message.

StaticUtilsTest.java: ADDED

git-svn-id: https://svn.forgerock.org/opendj/trunk@10202 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 8e8adac4   OPENDJ-1262 NPE in ChangeNumberIndex during server startup ... Browse Dir » 
    Code review: Matthew Swift

Caused by r10049.
Problem is down to the initialization sequence:
1. thread 1 - MultimasterReplication.initializeSynchronizationProvider()
1.1. it creates the ReplicationServerListener
1.1.1. the ReplicationServerListener in turn creates the ReplicationServer
1.1.1.1. the ReplicationServer in turn creates the ChangelogDB
1.1.1.1.1. the ChangelogDB in turn creates the ChangeNumberIndexer thread and STARTs it
1.1.1.1.1. the ChangelogDB starts the ChangeNumberIndexer thread
1.2. it proceeds with creating the LDAPReplicationDomain objects one by one
2. thread 2 - ChangeNumberIndexer.run()
2.1. it calls ChangeNumberIndexer.initialize()
2.1.1. ChangeNumberIndexer.initialize() calls MultimasterReplication.isECLEnabledDomain(baseDN)

Steps 1.2. and 2.1.1. are running concurrently.
If 2.1.1. is run before 1.2. is completed, In ChangeNumberIndexer.initialize():
1) MultimasterReplication.isECLEnabledDomain(baseDN) returns false, hence a cursor to the relevant replica DBs is not created
2) then the call to nextChangeForInsertDBCursor.getRecord() returns null, later throwing a NullPointerException because the ChangeNumberIndexer thread is in an illegal state: it was expecting to find an UpdateMsg with the correct CSN stamped on it.



MultimasterReplication.java:
Added State enum + state instance member to tell whether MultimasterReplication is ready for work.
Removed isRegistered instance member superseded by state instance member.
In isECLEnabledDomain(), completeSynchronizationProvider() and finalizeSynchronizationProvider(), deal with thread waits.

DomainFakeCfg.java:
Implemented toString().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10200 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 259bfb98   Fixed python exception in fractional-replication testsuite ... Browse Dir » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10198 41b1ffd8-f28e-4786-ab96-9950f0a78031
    csovant
     

13 Jan, 2014

3 commits

09 Jan, 2014

1 commit

  • c377bb85   Fixed broken tests ... Browse Dir » 
    ReplicationDomain.java:
Made ieContext private and used an AtomicReference for it + In acquireIEContext() and releaseIEContext(), removed now useless synchronized keyword.
In acquireIEContext(), returned the created ieContext + changed all client code to assign a variable from the result.

ReplicationDomainTest.java:
Avoid NPE in case the ieContext is released before the asserts.
Called ReplicationDomain.getImportExportContext().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10113 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

08 Jan, 2014

2 commits

  • ae2fae30   Increased encapsulation of ReplicationDomain.IEContext + moved more behaviour to it. ... Browse Dir » 
    Reduced members' visibilities in LDAPReplicationDomain + removed several methods.


ReplicationDomain.java:
Do not access the IEContext field several times in one method in case it gets updated in the middle, instead accessed it once at the top of a method and passed it down private method calls.
Moved importInProgress(), getTotalEntryCount(), getLeftEntryCount() to IEContext class.
Added getImportExportContext().

LDAPReplicationDomain.java:
Reduced visibility of many methods.
Removed followImport field + setFollowImport() which can be calculated from importErrorMessageId + added and used isFollowImport() instead.
Replaced calls to "retrievesBackend(getBaseDN())" with getBackend() + inlined retrievesBackend().
Called getImportExportContext() to access the IEContext.

FractionalLDIFImportPlugin.java:
Consequence of removing LDAPReplicationDomain.setFollowImport().
In doLDIFImport(), factorized code between branches of an if statement + extracted method isNotEmpty().
In flushFractionalConfigIntoEntry(), used StaticUtils.collectionToString() + extracted method add()

ReplicationMonitor.java:
In getMonitorData(), simplified the code.

ReplicationDomainTest.java:
Extracted methods waitEndExport(), assertExportSucessful(), buildExportedData().

*.java:
Updated copyright years for r10098.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10103 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 04664309   Replaced/removed numerous fields and methods of ReplicationDomain by directly storing the config. ... Browse Dir » 
    Changed a lot of subclasses due to this change.


ReplicationDomain.java:
Pulled up generationId, config fields, getGenerationID(), readAssuredConfig() and needReconnection() methods from LDAPReplicationDomain.
Replaced baseDN, serverID, groupId, refUrls, initWindow fields with new config field + encapsulated fields, updated getters, removed setters
Replaced assured, assuredMode, assuredSdLevel, assuredTimeout fields with new assuredConfig field + encapsulated fields, updated getters, removed setters
Removed domains field, never used.
Inlined stopDomain().
Added generationId to the ctor + implemented getGenerationID() + added setGenerationID().
Extracted method restartService(), needsAck().

LDAPReplicationDomain.java:
Pulled up generationId, config fields, getGenerationID(), readAssuredConfig() and needReconnection() methods to ReplicationDomain.
In ctor, consequence of the change to ReplicationDomain.

DSInfo.java:
Made it immutable.

StartSessionMsg.java:
Code cleanup.
Javadocs.


DummyReplicationDomain.java, FractionalReplicationTest.java, FakeReplicationDomain.java, FakeStressReplicationDomain.java:
Consequence of the changes to ReplicationDomain.
Pulled up generationId, getGenerationID(), setGeneration() to ReplicationDomain.
Called ReplicationDomain.getConfig().

TopologyViewTest.java:
Consequence of the change to ReplicationDomain.getRefUrls().
Extracted method checkLists().
Code cleanup.

AssuredReplicationServerTest.java:
Streamlined and simplified the createFakeReplicationDomain() methods + removed the boolean assured parameter.
Added getAssuredType(AssuredMode).
In FakeReplicationDomain, consequence of the changes to ReplicationDomain.
In checkUpdateAssuredParameters(), changed the assert a bit to match removal of the boolean assured parameter.
Removed useless calls to assertNotNull() after calling createReplicationServer() (result is never null).
Replaced newFakeCfg() by newDomainConfig().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10098 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

07 Jan, 2014

1 commit

03 Jan, 2014

6 commits

  • 2ae9d682   Fixed test failure for JEChangeNumberIndexDBTest.testClear(). ... Browse Dir » 
    Problem was due to the CNIndexDB state not being properly cleaned up.
I chose to rely on ReplicationTestCase.remove(ReplicationServer) to clean up the state of the ChangeNumberIndexDB.


JEChangelogDB.java:
Extracted getChangeNumberIndexDB(boolean) from getChangeNumberIndexDB().

JEChangeNumberIndexDB.java:
Removed trimDone field.
In shutdown(), used Thread.join().
In run(), removed duplicated code that I unfortunately added there in r9881.

JEChangeNumberIndexDBTest.java:
Replaced newCNIndexDB() with getCNIndexDBNoTrimming() + Removed createCleanDir() + relied on ReplicationTestCase.remove(ReplicationServer) to clean up the state of the ChangeNumberIndexDB.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10079 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 30f26b24   JEChangeNumberIndexDBTest.java: ... Browse Dir » 
    Fixed an infinite loop after r10071 (Why did it not fail when ran locally?) + extracted method assertOnlyNewestRecordIsLeft().
testClean() this test is still failing in CLI mode, or when you add a dependency towards method testTrim(). I haven't fully grasped yet why is that.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10078 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • d4a58ec3   ImportLDIFTestCase.java: ... Browse Dir » 
    Code cleanup.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10077 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • f5dd2d7b   OPENDJ-1205 Remove network layer from External ChangeLog implementation ... Browse Dir » 
    More code cleanup and encapsulation.

StartECLSessionMsg.java:
Introduced enums for ECLRequestType and Persistent.
Completed javadocs from javadocs coming from ECLServerHandler.

ECLServerHandler.java:
Replaced individual fields by directly storing the StartECLSessionMsg object.
Changed visibility to private in a few places.
Replaced isPersistent() by isNonPersistent() and getSearchPhase() by isInitPhaseDone().

ECLServerWriter.java
Used better encapsulated methods from ECLServerHandler.

ECLSearchOperation.java, SynchronizationMsgTest.java:
Consequence of the changes to StartECLSessionMsg.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10076 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 07240b05   Fixed a few test failures in ExternalChangeLogTest, broken after r10049. Created… ... Browse Dir » 
    … fake LDAP replication domains, so they can participate in the external changelog.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10073 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • dd4e5fcf   Fixed a few test failures in ExternalChangeLogTest, broken after r10049. Created… ... Browse Dir » 
    … fake LDAP replication domains, so they can participate in the external changelog.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10072 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

02 Jan, 2014

5 commits

  • e091af13   OPENDJ-1090 ECL changenumbers get reset after a purge and server restart ... Browse Dir » 
    
Never trim the newest (last) record in the ChangeNumberIndexDB from production code.
This last record contains the last published change number, which is the one that must be read on startup and used to compute the following monotonically increasing change numbers.


JEChangeNumberIndexDB.java:
Improved and fixed javadocs.
In ctor, factorized code doing twice the same thing. 
In addRecord(), updated the newestChangeNumber after a newest record is committed.
In clear(DN, AtomicBoolean), updated the oldestChangeNumber + do not trim the newest record from the DB.

JEChangeNumberIndexDBTest.java:
Updated testClear() to assert the newest record is never trimmed by production code.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10071 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 29e3f086   ExternalChangeLogTest.java: ... Browse Dir » 
    Updated copyright.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10070 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 54887227   Fixed ExternalChangeLogTest.ECLReplicationServerTest(), broken after r10049. Cre… ... Browse Dir » 
    …ated a fake domain for it, so it participates in the external changelog.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10069 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 5524cd94   OPENDJ-1262 NPE in ChangeNumberIndex during server startup ... Browse Dir » 
    ChangeNumberIndexer.java:
Removed useless boolean parameter from ensureCursorExists().
Changed visibility of getPrecedingCSN() to unit test it.

ChangeNumberIndexerTest.java:
Added tests for ChangeNumberIndexer.getPrecedingCSN().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10067 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • c0d8ec30   Added tests for BoundedWorkQueueStrategy (finally). ... Browse Dir » 
    BoundedWorkQueueStrategy.java:
Extracted calls to undesired DirectoryServer static methods into overridable methods.

BoundedWorkQueueStrategyTest.java: ADDED

git-svn-id: https://svn.forgerock.org/opendj/trunk@10066 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

23 Dec, 2013

2 commits

  • a7a126d3   Fix feature envy between ListenerThread and ReplicationDomain ... Browse Dir » 
    ListenerThread was 100% implemented in terms of ReplicationDomain method calls. It should therefore be part of ReplicationDomain.

* inline ListenerThread into ReplicationDomain and use new DirectoryThread lifecycle methods
* remove shutdown parameter from ReplicationDomain.processUpdate()
* reduce visibility of ReplicationDomain.receive()
* inline ReplicationDomain.processUpdateDoneSynchronous().


git-svn-id: https://svn.forgerock.org/opendj/trunk@10054 41b1ffd8-f28e-4786-ab96-9950f0a78031
    matthew
     
  • 1089ff03   OPENDJ-1263 Changenumber does not progress on the second replication server of a topology ... Browse Dir » 
    
The medium consistency point was not progressing on DSRS 2 because the ChangeNumberIndexer mediumConsistencyCSN had been polluted by CSN from "cn=admin data".
Problem is that ChangeNumberIndexer's publishUpdateMsg() was filtering out updates from ECL disabled domains, but not from domains unknown to ECL.
On start up, update messages can be received from replication domains have not been been configured yet on the current replication server.
The fix consists in only updating the medium consistency point with update messages or replica heartbeats coming from explicitly enabled replication domains.
After this fix, changes from ECL disabled domains will still be stored in dedicated replicaDBs, there is no change in this functionality.


MultimasterReplication.java
Renamed isECLDisabledDomain() to isECLEnabledDomain() so it caters better for the 3 states of the domains: enabled, disabled and unknown
Code cleanup.

ChangeNumberIndexer.java:
Extracted method isECLEnabledDomain() to use as a seam for unit testing which calls to MultimasterReplication.isECLEnabledDomain().

ChangeNumberIndexerTest.java
Added one test emptyDBThreeInitialDSsOneIsNotECLEnabledDomain().
Renamed assertAddedRecords() to assertExternalChangelogContent(), startIndexer() to startCNIndexer(), stopIndexer() to stopCNIndexer() and indexer to cnIndexer.
In startCNIndexer(), overrode ChangeNumberIndexer.isNotECLEnabledDomain().

AttributeTypeConstants.java:
Added support for "cn"


git-svn-id: https://svn.forgerock.org/opendj/trunk@10049 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

20 Dec, 2013

1 commit

  • bb139260   Tests cleanup. ... Browse Dir » 
    
ReplicationTestCase.java
Removed emptyOldChanges parameter from openReplicationSession() methods because it has no effect whatsoever.

ReplicationServerTest.java:
Overrode openReplicationSession() to also check the broker is connected.

InitOnLineTest.java:
Stopped constantly calling DN.decode() and used instance variable instead.

*.java:
Consequence of the changes to ReplicationTestCase.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10040 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac