org.forgerock / org.forgerock.opendj

14 May, 2014

4 commits

  • 604a356f   Code cleanup. ... Browse Code » 
    LDAPReplicationDomain.java:
Removed generator field, duplicated with super class.
Extracted method getFirstResult(InternalSearchOperation).
In findAndRenameChild(), removed useless null check.

PersistentServerState.java
Changed visibilities to default or private.
Removed conn field + used static import instead.
Removed on ctor only used by tests.
In save() simplified the code a lot.
In checkAndUpdateServerState(), reduced local variable scope + renamed a few of them.
Extracted method getFirstResult(InternalSearchOperation).

PersistentServerStateTest.java:
Consequence of removing one ctor from PersistentServerState.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10727 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 98095d4b   Fix OPENDJ-1358: Backup task logs path in ID field, and ID in path field ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10724 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     
  • 7d82b6bc   Code cleanup. ... Browse Code » 
    
LDAPReplicationDomain.java:
Changed numReplayedPostOpCalled field from int to AtomicInteger.
Extracted methods newSet(), newList(), exists(), canRemoveAttribute().
Inlined getPendingUpdatesCount(), getNumReplayedPostOpCalled(), getNumResolvedModifyConflicts(), getNumResolvedNamingConflicts() and getNumUnresolvedNamingConflicts().
In contains(), changed parameter names.
Added javadocs to instance variables.
Reduced visibilities to default or private.
Added braces to if statements.
Used static imports.
Fixed Eclipse warnings.
Replaced unnecessary use of MessageBuilder by adding arguments to replication messages.

replication*.properties:
Modified messages to include arguments.

PendingChanges.java:
Updated copyright year after r10722.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10723 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 029a42ad   Code cleanup. ... Browse Code » 
    PendingChanges.java:
Removed unused commit(CSN).
In remove(CSN), now return void.
Reduced visibilities to default or private.
Used final keyword.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10722 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

13 May, 2014

3 commits

12 May, 2014

1 commit

08 May, 2014

1 commit

07 May, 2014

1 commit

  • 53efbe59   OPENDJ-1388 – Implement simple changelog db based on single log file ... Browse Code » 
    * Add new package org.opends.server.replication.server.changelog.file 
 containing the log file implementation 

* The new package contains the following classes :
** Classes implementing the log - independant from changelog and built from scratch
  - LogFile : implementation of log based on a single file
  - LogWriter : writer for the log
  - LogReaderPool : pool of readers for the log
  - Record : a (key, value) log record
  - RecordParser : interface for the convertion from record to bytes and bytes to record
  - DecodingException : exception thrown when record decoding fails
** Classes implementing the changelog API, based on the log implementation. 
 - FileChangelogDB
 - FileChangeNumberIndexDB
 - FileChangeNumberIndexDBCursor
 - FileReplicaDB
 - FileReplicaDBCursor
 - ReplicationEnvironment

* Add unit tests for org.opends.server.replication.server.changelog.file package

* Add new attribute 'ds-cfg-replication-db-implementation' in 
 ds-cfg-replication-server (ServerReplicationConfig class), allowing
 to choose the db implementation to use for changelog in directory server: 
 either je or log file. Default is 'je'.

* Add new option 'org.opends.test.replicationDbImpl' for test target in build.xml
 to choose the db implementation to for changelog when running the tests.
 Default is 'log' (log file).

* Update all replication unit tests to allow the selection of changelog db implementation
 to use in tests.
 
* Fix the ReferentialIntegerityPluginTestCase to do better cleanup at end of test in
 order to avoid side-effect on other tests.



git-svn-id: https://svn.forgerock.org/opendj/trunk@10713 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

06 May, 2014

1 commit

05 May, 2014

2 commits

01 May, 2014

1 commit

  • 21b0e742   Fix various bugs in access log filtering: ... Browse Code » 
    * OPENDJ-1429: Access log filtering criterion request-target-dn-not-equal-to does not work
* OPENDJ-1432: Access log filtering criterion on etime property does not work
* OPENDJ-1433: Access log filtering criterion on result code property does not work
* OPENDJ-1434: Access log filtering criteria user-dn-not-equal-to does not work
* OPENDJ-1435: Access log filtering criteria user-is-not-member-of does not work.

Also added unit tests. Use a single commit because a) the fixes are very small and closely related, and b) some of the bugs have a common cause.


git-svn-id: https://svn.forgerock.org/opendj/trunk@10705 41b1ffd8-f28e-4786-ab96-9950f0a78031
    matthew
     

30 Apr, 2014

5 commits

  • 83d0c476   OPENDJ-1454 Remove all code supporting "dc=replicationChanges" ... Browse Code » 
    Done. Removed a lot of code.
Noticeable changes:


ReplicationCliMain.java:
Extracted a few methods to factorize code.

GenerationIdTest.java:
Please note the strange case of checkChangelogSize().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10703 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • e853ad36   OPENDJ-1259 Make the Medium Consistency Point support replicas temporarily leaving the topology ... Browse Code » 
    Preliminary work, also needed by Nicolas for OPENDJ-1177.


ByteSequenceReader.java:
Added peek() and peek(int offset).

ByteSequenceReaderTest.java:
Added a test.
Added method b() to ease reading in int -> byte conversion.


git-svn-id: https://svn.forgerock.org/opendj/trunk@10696 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 252b2b9f   Checkpoint for OPENDJ-1343 Migrate dsconfig ... Browse Code » 
    - Removed System.getProperty("user.dir") from the initialization of the configuration framework and replaced it with the System.getenv(["INSTALL_ROOT"]["INSTANCE_ROOT"]).

git-svn-id: https://svn.forgerock.org/opendj/trunk@10689 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette
     
  • 7fd5dd6b   OPENDJ-1259 (CR-3443) Make the Medium Consistency Point support replicas tempora… ... Browse Code » 
    …rily leaving the topology

Changed ChangeTimeHeartbeatMsg to be able to send timestamps for a replica stopping. This is done by adding an event type to cover both normal heartbeat and temporarily offline replica.
This allows RSs to compute medium consistency accurately by using the stop time from the originating server.
Because of this change, incremented protocol version to v8.



ProtocolVersion.java:
Added REPLICATION_PROTOCOL_V8.
Fixed javadocs by using lists.

ByteArrayBuilder.java, ByteArrayScanner.java, ByteArrayTest.java: ADDED

ChangeTimeHeartbeatMsg.java:
Added eventType field + added isReplicaOfflineMsg().
Made ctor private + added factory methods heartbeatMsg() and replicaOfflineMsg().
Used ByteArrayBuilder and ByteArrayScanner.

CTHeartbeatPublisherThread.java:
In run(), on shutdown, handle sending a replica offline message.



SynchronizationMsgTest.java:
Added changeTimeHeartbeatMsgTest() and assertCTHearbeatMsg().
Code cleanup + used static import for ProtocolVersion.



DataServerHandler.java:
Consequence of the change to StopMsg ctor.
Extracted method publishStopMsg() for increased readability.

ReplicationServerDomain.java:
In processChangeTimeHeartbeatMsg(), added support for offline replica messages + declared DirectoryException checked exception.

ServerHandler.java:
Consequence of the change to ReplicationServerDomain.processChangeTimeHeartbeatMsg() checked exception.



ChangeNumberIndexer.java:
In initialize(), handle offline replicas.
In moveForwardMediumConsistencyPoint(), now throw ChangelogException.
In removeCursor(), now call resetNextChangeForInsertDBCursor().

ChangeNumberIndexerTest.java:
Expanded the emptyDBTwoDSsOneGoingOffline() test.



JEChangelogDB.java, ReplicationDomainDB.java:
In replicaOffline(), now throw ChangelogException and call ReplicationDbEnv.addOfflineReplica()
Added ChangelogDBPurger.initiateShutdown().

ReplicationDbEnv.java:
Added OFFLINE_TAG constant, toReplicaOfflineEntry(), addOfflineReplica() and putInChangelogStateDB(), toByteArray()().
In decodeChangelogState(), handle offline replica information.
Changed several method signature to use Entry<byte[], byte[]> instead of Entry<String, String>.

ReplicationDbEnvTest.java:
Fixed indentation + added tests.

ChangelogState.java:
Added offlineReplicas field + addOfflineReplica() and getOfflineReplicas() + changed toString() to display it.



StopMsg.java:
Added toString().

JEReplicaDBCursor.java:
Improved comments and javadocs.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10688 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • cedb0039   Save eclipse workspace configuration changes ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10687 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

29 Apr, 2014

5 commits

  • 280f110c   Checkpoint for OPENDJ-1343 Migrate dsconfig ... Browse Code » 
    - Fixed copyright(s) and replaced OID for fullVendorVersion to 1.3.6.1.4.1.36733.2.1.1.141

git-svn-id: https://svn.forgerock.org/opendj/trunk@10686 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette
     
  • 1341b1cc   Checkpoint for OPENDJ-1343 Migrate dsconfig ... Browse Code » 
    ** Dsconfig.java
- Removed FiXMe and unused comment.

** dsconfig.properties
- Fixed typo.

** BuildVersion.java
- gets now the version from the "fullVendorVersion" (RootDSEBackend) instead of the cn=monitor.

** CoreSchema.java
- Added FULL_VENDOR_VERSION_ATTRIBUTE_TYPE.

** RootDSE.java
- Added ATTR_FULL_VENDOR_VERSION.
- Added getFullVendorVersion().
- Modified SEARCH_REQUEST to add the ATTR_FULL_VENDOR_VERSION.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10683 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette
     
  • 0f5615c0   Support suffixes containing space in the changelog state DB. ... Browse Code » 
    
ReplicationDbEnvTest.java:
Added one test case.

ReplicationDbEnv.java:
Fixed the code.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10682 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • cea91160   Checkstyle ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10681 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 413f8717   Added unit tests for encoding/decoding changelog state DB entries. ... Browse Code » 
    
ReplicationDbEnv.java:
Split readChangelogState() into decodeChangelogState() and readWholeState().
Changed buildServerIdKey() into toReplicaEntry().
Combined buildGenIdKey() and buildGenIdData() into toGenIdEntry().
Made some methods package private for testing.
Made some methods protected for easing unit testing.
Extracted openJEEnvironment() from ctor.
Renamed getOrAddDb() to getOrAddReplicationDB().

ReplicationDbEnvTest.java: ADDED

ReplicationDB.java:
Consequence of the change to ReplicationDbEnv.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10680 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

28 Apr, 2014

2 commits

  • 9d668e38   Checkstyle... *sigh* ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10679 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 33c09279   Code cleanup: ... Browse Code » 
    - Increased MessageHandler encapsulation.
- Removed ServerHandler.closeSession() because it is adding unnecessary complexity in understanding calling sites.


MessageHandler.java:
Reduced class visibility to package private. Did the same for a few methods.
Made several fields private for better encapsulation.
Renamed getDomain() to setDomain().
In setBaseDNAndDomain(), changed the code to always set the ReplicationServerDomain, so it can never be null. Avoids any NPE + allows to remove null checks.

DataServerHandler.java:
In receiveNewStatus(), consequence of the change to MessageHandler: removed null check on replicationServerDomain.

ECLServerHandler.java, ReplicationServerHandler.java:
Consequence of the change to MessageHandler: removed null check on replicationServerDomain.

ServerHandler.java:
Consequence of the change to MessageHandler: removed null check on replicationServerDomain.
Removed closeSession() because it is adding unnecessary complexity in understanding calling sites.
In abortStart(), inlined the code of closeSession().
Removed the unneeded overriding methods for getInCount() and getOutCount().

ReplicationServer.java:
Consequence of removing ServerHandler.closeSession().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10678 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

25 Apr, 2014

3 commits

  • 9a71cb2f   Checkpoint for OPENDJ-1343 Migrate dsconfig ... Browse Code » 
    - Fixed checkstyle errors on DSConfig and LDAPManagementContextFactory.



git-svn-id: https://svn.forgerock.org/opendj/trunk@10677 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette
     
  • 49ae9dd0   Checkpoint for OPENDJ-1343 Migrate dsconfig ... Browse Code » 
    - Moved dsconfig package to opendj-config.
-> Several checkstyle errors still exist and will be fixed after this commit in order to let the changes on main files more readable.
-> BuildVersion.java will be renamed, moved later.


** DsConfig.java
- Added logger.
- Removed checkVersionMismatch and moved it to LDAPManagementContextFactory where the connection is opened.

** BuildVersion.java
- Added function to get the current configuration entry value of "cn=Version,cn=monitor"

** LDAPManagementContextFactory.java
- Added checkVersionMismatch.

**config.properties
- Added messages linked to BuildVersion.

** SDK *check* style modifications on :
- PropertyEditorModification.java
- PropertyValueEditor.java
- HelpSubCommandHandler.java
- SetPropSubCommandHandler.java
- SubCommandHandler.java
- ListSubCommandHandler.java
- CreateSubCommandHandler.java
- PropertyValuePrinter.java
- DeleteSubCommandHandler.java
- SubCommandHandlerFactory.java
- ArgumentExceptionFactory.java
- GetPropSubCommandHandler.java
- CLIProfile.java

** pom file : added dependency on opendj-cli.



git-svn-id: https://svn.forgerock.org/opendj/trunk@10676 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette
     
  • 7c3389c3   Typo in link ... Browse Code » 
    Thanks, Chris Ridd!

git-svn-id: https://svn.forgerock.org/opendj/trunk@10675 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     

24 Apr, 2014

3 commits

23 Apr, 2014

2 commits

  • be9a267d   OPENDJ-1259 Make the Medium Consistency Point support replicas temporarily leaving the topology ... Browse Code » 
    Preliminary work, also needed by Nicolas for OPENDJ-1177.


ByteSequenceReader.java:
Added peek() and peek(int offset).

ByteSequenceReaderTest.java:
Added a test.
Added method b() to ease reading in int -> byte conversion.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10670 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • e8735348   OPENDJ-1448 Remove JReplicaDB flushing thread and msg queue ... Browse Code » 
    It seems unnecessary to have a msgQueue on top of JE since JE's already has a builtin cache that handles the same responsibility.
This improvement removes the JEReplicaDB.msgQueue and the associated flushing thread to save on memory and resources.

Code cleanup in JEChangeNumberIndexDB after CR-3388.


JEReplicaDB.java:
Does not implement Runnable anymore.
Removed fields msgQueue, queueSizeBytes, queueMaxBytes, thread, flushLock.
Added and used shutdown field to compensate for removing the thread field.
Removed methods collectAllPermits(), flush(), run() and stop().
In shutdown(), used AtomicBoolean.compareAndSet().

ReplicationDB.java:
Renamed addEntries(List<UpdateMsg>) to addEntry(UpdateMsg).

JEReplicaDBTest.java:
Removed now unnecessary waitChangesArePersisted().


JEChangelogDB.java:
In shutdown(), enforced threads joining + called Thread.interrupt() to ensure shutdown. This prevents message about unclosed cursors in integrated unit tests.

ReplicationServer.java:
Removed getQueueSize().


JEChangeNumberIndexDB.java:
Removed unused oldestChangeNumber.
In shutdown(), used AtomicBoolean.compareAndSet() + removed useless call to notify().

JEChangeNumberIndexDBTest.java:
Fixed javadocs.


ChangeTimeHeartbeatMsg.java:
Implemented toString().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10669 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

18 Apr, 2014

1 commit

  • fe2017f0   OPENDJ-1430 Some changes are missing from the external changelog changeNumber ... Browse Code » 
    The bug was due to a very complex interaction between various components. Here is a scenario and explanation:
1- the change number indexer has no more records to proceed, because all cursors are exhausted, so it calls wait().
2- a new change Upd1 comes in for an exhausted cursor, medium consistency cannot move.
3- a new change Upd2 comes in for a cursor that is not already opened, medium consistency can move, so wake up the change number indexer.
3- on wake up, the change number indexer calls next(), advancing the CompositeDBCursor, which recycles the exhausted cursor, then calls next() on it, making it lose its change. CompositeDBCursor currentRecord == Upd1.
4- on the next iteration of the loop in run(), a new cursor is created, triggering the creation of a new CompositeDBCursor => Upd1 is lost. CompositeDBCursor currentRecord == Upd2.

The problem comes from two parts:
- CompositeDBCursor consumes next change from a cursor (which completely forget about this change) and stores it itself
- ChangeNumberIndexer manages recycling/creating cursors on its own and recreates CompositeDBCursor when a new cursor is created.

The fix required:
- Preventing CompositeDBCursor from consuming changes from underlying cursors until it can forget about this same change itself.
- Ensuring only ChangeNumberIndexer handle recycling the cursors it owns instead of having both CompositeDBCursor and ChangeNumberIndexer trying to do it. It is also more performant to let ChangeNumberIndexer manage its cursors.



CompositeDBCursor.java:
Added recycleExhaustedCursors field to tell the composite whether it can recycle the cursors itself or not (recycling the cursors is currently needed for persistent searches on the changelog, maybe will we be able to remove it in the future, that would simplify the code a lot).
Modified the ctor to pass in value of recycleExhaustedCursors.
Removed currentRecord and currentData fields, replaced by reading the record and field on the first entry in the cursors SortedMap.
Added state field to ensure the first call to next() does not consume the first change in the cursors SortedMap.

ChangeNumberIndexer.java:
ChangeNumberIndexer now manages alone the cursors recycling and creation and recreates the CompositeDBCursor when needed.
In run(), removed the now unneeded call to next() after the wait.
Added recycleExhaustedCursors().

JEChangelogDB.java:
Consequence of the change to CompositeDBCursor. Kept old recycling behaviour.


ChangeNumberIndexerTest.java:
Added emptyDBTwoDSsDoesNotLoseChanges() to cover the case being fixed by current commit.
Renamed test methods dropping the "Initial" when it was not adding much to the test comprehension.

CompositeDBCursorTest.java:
In newUpdateMsg(), added toString() implementation to help debug.
Removed recycleTwoElementCursorsTODOJNR().
In recycleTwoElementCursors(), changed the tests a bit to match the changes to CompositeDBCursor.


git-svn-id: https://svn.forgerock.org/opendj/trunk@10667 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

17 Apr, 2014

2 commits

  • 3fb4923f   OPENDJ-1440 On startup, change number can progress without waiting for any heart… ... Browse Code » 
    …beat from known replicas

On startup, if a replication server knows about several replicas, then it must wait to receive some sort of alive information for each of them before being able to move the medium consistency forward.
Changes or heartbeats received after replication server started are acceptable.
Likewise, changes that would have been received before the replication server stopped are also acceptable.

This was fixed on replication server startup, by initializing the lastAliveCSN for each known replica, with the oldest possible CSN (timestamp == 0).
Then when checking if the medium consistency can move forward, if no medium consistency is set, then the lastAliveCSN for each known replica must have a timestamp != 0.


ChangeNumberIndexer.java:
In canMoveForwardMediumConsistencyPoint(), call allInitialReplicasArePastOldestPossibleCSN() if the medium consistency CSN is not set.
Added methods oldestPossibleCSN(), allInitialReplicasAreAlive().

ChangeNumberIndexerTest.java:
In emptyDBTwoInitialDSs(), slightly modified the code to test current bug.
In startCNIndexer(), added the initial ECL enabled domains as a parameter.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10666 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 8d301231   OPENDJ-1439 Change number stops progressing with cross domain replication ... Browse Code » 
    The changeNumber progression was blocked when the mediumConsistencyCSN was from a different baseDN than the new change to add to the changeNumber index DB.
Fixed this problem by also storing the baseDN of the mediumConsistencyCSN in the ChangeNumberIndexer class.


ChangeNumberIndexer.java:
Renamed mediumConsistencyCSN to mediumConsistency and changed its type from CSN to Pair<DN, CSN>.
In tryNotify(), removed the now useless baseDN parameter (replaced with mediumConsistency field).
Improved comments.

ChangeNumberIndexerTest.java:
Renamed BASE_DN to BASE_DN1.
Added BASE_DN2 and test emptyDBTwoInitialDSsDifferentDomains().
In assertExternalChangelogContent(), did some renaming.

SequentialDBCursor.java:
Improved toString().

ChangeNumberIndexDB.java:
Removed obsolete FIXME.


git-svn-id: https://svn.forgerock.org/opendj/trunk@10665 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

15 Apr, 2014

2 commits

  • ad679023   OPENDJ-1430 - Some changes are missing from the external changelog ... Browse Code » 
    Small enhancements / cleanups.


ChangeNumberIndexer.java:
Always updated lastAliveCSNs as the very last thing because it is used to decide whether the medium consistency point can move forward.


AddMsg.java:
In toString(), renamed the incorrectly named "changeNumber" attribute to "csn" + merged V1 and V2 protocol paths.
Extracted method encodeAttributes().
javadoc cleanup.

DeleteMsg.java, ModifyDNMsg.java, ModifyMsg.java:
In toString(), renamed the incorrectly named "changeNumber" attribute to "csn" + merged V1 and V2 protocol paths.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10662 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 52808d89   Code cleanup on from(Scope) - Replaced if...else.. by the switch(enum). ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10661 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette
     

14 Apr, 2014

1 commit

  • 2e440844   OPENDJ-1373 ... Browse Code » 
    - Added Scope converter.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10658 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette