incubator-crail.git
2 days ago[maven-release-plugin] prepare for next development iteration master
Jonas Pfefferle [Thu, 15 Nov 2018 16:02:45 +0000 (17:02 +0100)] 
[maven-release-plugin] prepare for next development iteration

2 days ago[maven-release-plugin] prepare release v1.1-rc6 v1.1-rc6
Jonas Pfefferle [Thu, 15 Nov 2018 16:02:18 +0000 (17:02 +0100)] 
[maven-release-plugin] prepare release v1.1-rc6

2 days ago[Build] include CREDITS and licenses in META-INF
Jonas Pfefferle [Thu, 15 Nov 2018 15:55:36 +0000 (16:55 +0100)] 
[Build] include CREDITS and licenses in META-INF

Include CREDITS and licenses in META-INF.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-84

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 days ago[Build] Include CREDITS,README and licenses in bin
Jonas Pfefferle [Thu, 15 Nov 2018 15:43:22 +0000 (16:43 +0100)] 
[Build] Include CREDITS,README and licenses in bin

Include CREDITS, README and licenses in binary tarball.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-83
https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-82

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 days ago[Documentation] Change README to be used in binary
Jonas Pfefferle [Thu, 15 Nov 2018 15:37:23 +0000 (16:37 +0100)] 
[Documentation] Change README to be used in binary

Adapt README that it can also be used in the binary tarball.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-83

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 days ago[maven-release-plugin] prepare for next development iteration
Jonas Pfefferle [Tue, 13 Nov 2018 10:29:26 +0000 (11:29 +0100)] 
[maven-release-plugin] prepare for next development iteration

4 days ago[maven-release-plugin] prepare release v1.1-rc5 v1.1-rc5
Jonas Pfefferle [Tue, 13 Nov 2018 10:28:18 +0000 (11:28 +0100)] 
[maven-release-plugin] prepare release v1.1-rc5

4 days ago[Build] rat-check handle licenses of dependencies
Jonas Pfefferle [Tue, 13 Nov 2018 10:21:12 +0000 (11:21 +0100)] 
[Build] rat-check handle licenses of dependencies

Handle licenses of depenendcies in rat-check.

https://jira.apache.org/jira/browse/CRAIL-82

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 days ago[Documentation] update HISTORY
Jonas Pfefferle [Tue, 13 Nov 2018 10:05:23 +0000 (11:05 +0100)] 
[Documentation] update HISTORY

Update HISTORY file.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
5 days ago[LICENSE] add licenses of dependencies
Jonas Pfefferle [Mon, 12 Nov 2018 10:24:31 +0000 (11:24 +0100)] 
[LICENSE] add licenses of dependencies

Add licenses of all dependencies and their copyrights.

https://jira.apache.org/jira/browse/CRAIL-81

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 days agoMoving to DiSNI v2.0 and DaRPC v1.8
Patrick Stuedi [Sun, 11 Nov 2018 20:13:54 +0000 (21:13 +0100)] 
Moving to DiSNI v2.0 and DaRPC v1.8

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-79
https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-80

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
12 days ago[maven-release-plugin] prepare for next development iteration
Jonas Pfefferle [Mon, 5 Nov 2018 15:07:29 +0000 (16:07 +0100)] 
[maven-release-plugin] prepare for next development iteration

12 days ago[maven-release-plugin] prepare release v1.1-rc4 v1.1-rc4
Jonas Pfefferle [Mon, 5 Nov 2018 15:06:58 +0000 (16:06 +0100)] 
[maven-release-plugin] prepare release v1.1-rc4

12 days ago[Documentation] update HISTORY
Jonas Pfefferle [Mon, 5 Nov 2018 13:46:11 +0000 (14:46 +0100)] 
[Documentation] update HISTORY

Update HISTORY.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
12 days ago[Documentation] include doc generation in README
Jonas Pfefferle [Mon, 5 Nov 2018 13:40:24 +0000 (14:40 +0100)] 
[Documentation] include doc generation in README

Include description how to generate documentation in README.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
12 days ago[Build] base directory for bin tarball
Jonas Pfefferle [Mon, 5 Nov 2018 13:23:20 +0000 (14:23 +0100)] 
[Build] base directory for bin tarball

Base directory in bin tarball:

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-74

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
12 days ago[Build] bin tarball: NOTICE, DISCLAIMER, LICENSE
Jonas Pfefferle [Mon, 5 Nov 2018 13:11:23 +0000 (14:11 +0100)] 
[Build] bin tarball: NOTICE, DISCLAIMER, LICENSE

Include NOTICE, DISCLAIMER, LICENSE in binary tarball.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
12 days ago[Documentation] JIRA-61 updating the documentation
Animesh Trivedi [Mon, 5 Nov 2018 12:54:12 +0000 (13:54 +0100)] 
[Documentation] JIRA-61 updating the documentation

Removing reference to "crail-1.0" string in the
documentation. All referneces are set to the
$CRAIL_HOME.

Signed-off-by: Animesh Trivedi <atrivedi@apache.org>
12 days ago[Documentation] JIRA-61 updating the documentation
Animesh Trivedi [Mon, 5 Nov 2018 12:46:41 +0000 (13:46 +0100)] 
[Documentation] JIRA-61 updating the documentation

Three corrections
- Fix path in the NOTE of building code
- Fix typo SRD vs SSD
- Clarify that if a IPMC vote fails/is canceled you need to start over
  with a PPMC vote.
- IPMC vote template misses link to HISTORY and nexus staging.

Signed-off-by: Animesh Trivedi <atrivedi@apache.org>
12 days ago[Build] meta-inf: LICENSE, NOTICE and DISCLAIMER
Jonas Pfefferle [Mon, 5 Nov 2018 12:24:29 +0000 (13:24 +0100)] 
[Build] meta-inf: LICENSE, NOTICE and DISCLAIMER

Include LICENSE, NOTICE and DISCLAIMER in meta-inf for
binary release.

https://jira.apache.org/jira/browse/CRAIL-77

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 weeks ago[Build] Append filename to checksum files
Jonas Pfefferle [Tue, 30 Oct 2018 09:07:25 +0000 (10:07 +0100)] 
[Build] Append filename to checksum files

Append filename to checksum in checksum files. This allows to
check checksum with sha512sum -c <filename>

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-73

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[maven-release-plugin] prepare for next development iteration
Jonas Pfefferle [Fri, 26 Oct 2018 10:51:31 +0000 (12:51 +0200)] 
[maven-release-plugin] prepare for next development iteration

3 weeks ago[maven-release-plugin] prepare release v1.1-rc3 v1.1-rc3
Jonas Pfefferle [Fri, 26 Oct 2018 10:50:59 +0000 (12:50 +0200)] 
[maven-release-plugin] prepare release v1.1-rc3

3 weeks ago[Documentation] release process improved
Jonas Pfefferle [Fri, 26 Oct 2018 10:40:04 +0000 (12:40 +0200)] 
[Documentation] release process improved

Release process does not require to create checksum files
manually anymore.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-56
https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-61

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Build] Generate checksum files
Jonas Pfefferle [Thu, 25 Oct 2018 13:50:02 +0000 (15:50 +0200)] 
[Build] Generate checksum files

Generate checksum SHA-512 files for release tarballs.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-56

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Build] Generate correct source tarball
Jonas Pfefferle [Thu, 25 Oct 2018 13:43:14 +0000 (15:43 +0200)] 
[Build] Generate correct source tarball

Disable generation of source tarball inherited from Apache parent pom
since the name of the artifact does not adhere to
apche-crail-X.Y-incubating-src. Instead generate source tarball in
the assembly module. Add excludes to avoid committing build files.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-56

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Documentation] updating the copy step as JIRA-56 is partially fixed.
Animesh Trivedi [Thu, 25 Oct 2018 12:51:42 +0000 (14:51 +0200)] 
[Documentation] updating the copy step as JIRA-56 is partially fixed.

this updates JIRA-61
https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-61

Signed-off-by: Animesh Trivedi <atrivedi@apache.org>
3 weeks ago[Build] fix binary tarball name
Jonas Pfefferle [Thu, 25 Oct 2018 12:41:34 +0000 (14:41 +0200)] 
[Build] fix binary tarball name

Change name of binary tarball to adhere to
apche-crail-X.Y-incubating-bin

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-56

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Build] fix source tarball name
Jonas Pfefferle [Thu, 25 Oct 2018 12:15:12 +0000 (14:15 +0200)] 
[Build] fix source tarball name

Change name of source tarball to adhere to
apche-crail-X.Y-incubating-src

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-56

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Documentation] add license, revert rat exclude
Jonas Pfefferle [Thu, 25 Oct 2018 11:52:29 +0000 (13:52 +0200)] 
[Documentation] add license, revert rat exclude

Add Apache license to documentation and do not exclude
doc folder from rat check anymore.

https://jira.apache.org/jira/browse/CRAIL-71

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Documentation] JIRA-61 more details about the release process
Animesh Trivedi [Wed, 24 Oct 2018 11:12:36 +0000 (13:12 +0200)] 
[Documentation] JIRA-61 more details about the release process

1. Adding -Darguments="-DskipTests" in the release:perform step

2. Removing the additional assembling directory, and showing to
assemble everything directly in the SVN staging directory marked
as SSD

3. Template for the cancel vote email

Signed-off-by: Animesh Trivedi <atrivedi@apache.org>
3 weeks ago[Git] add gitignore
Jonas Pfefferle [Wed, 24 Oct 2018 08:45:32 +0000 (10:45 +0200)] 
[Git] add gitignore

Add gitignore to avoid accidentially adding binaries,
IDE files, etc.

https://jira.apache.org/jira/browse/CRAIL-69

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Client] check if CRAIL_HOME is set/empty
Jonas Pfefferle [Tue, 23 Oct 2018 14:26:22 +0000 (16:26 +0200)] 
[Client] check if CRAIL_HOME is set/empty

Check if environment variable CRAIL_HOME is set or empty.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-64

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Documentation] clarify where to find tarball
Jonas Pfefferle [Tue, 23 Oct 2018 14:43:46 +0000 (16:43 +0200)] 
[Documentation] clarify where to find tarball

Clarify build instructions where to find tarball after a
successful build.

https://jira.apache.org/jira/browse/CRAIL-67

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Documentation] fix path to crail script
Jonas Pfefferle [Tue, 23 Oct 2018 14:18:10 +0000 (16:18 +0200)] 
[Documentation] fix path to crail script

Fix path to crail script. The correct path is $CRAIL_HOME/bin/crail

https://jira.apache.org/jira/browse/CRAIL-66

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[maven-release-plugin] prepare for next development iteration
Jonas Pfefferle [Mon, 22 Oct 2018 12:54:09 +0000 (14:54 +0200)] 
[maven-release-plugin] prepare for next development iteration

3 weeks ago[maven-release-plugin] prepare release v1.1-rc2 v1.1-rc2
Jonas Pfefferle [Mon, 22 Oct 2018 12:53:33 +0000 (14:53 +0200)] 
[maven-release-plugin] prepare release v1.1-rc2

3 weeks ago[History]: rc2 fixes and improvements
Jonas Pfefferle [Mon, 22 Oct 2018 12:48:13 +0000 (14:48 +0200)] 
[History]: rc2 fixes and improvements

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 weeks ago[Documentation]: fixing JIRA-61
Animesh Trivedi [Mon, 22 Oct 2018 12:31:19 +0000 (14:31 +0200)] 
[Documentation]: fixing JIRA-61

write up about the release process.

Signed-off-by: Animesh Trivedi <atrivedi@apache.org>
3 weeks agoDocker: fix base image url/tag and DiSNI version
Jonas Pfefferle [Mon, 22 Oct 2018 08:42:21 +0000 (10:42 +0200)] 
Docker: fix base image url/tag and DiSNI version

Fix base image url/tag and update DiSNI version to match version
in pom file.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-58

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks ago[maven-release-plugin] prepare for next development iteration
Jonas Pfefferle [Thu, 18 Oct 2018 14:50:16 +0000 (16:50 +0200)] 
[maven-release-plugin] prepare for next development iteration

4 weeks ago[maven-release-plugin] prepare release v1.1-rc1 v1.1-rc1
Jonas Pfefferle [Thu, 18 Oct 2018 14:49:41 +0000 (16:49 +0200)] 
[maven-release-plugin] prepare release v1.1-rc1

4 weeks agoDocumentation: Add programming documentation
Jonas Pfefferle [Thu, 18 Oct 2018 14:30:42 +0000 (16:30 +0200)] 
Documentation: Add programming documentation

Add documentation how to program against Crail.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-55

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks agoHISTORY: bug fixes and improvements of v1.1
Jonas Pfefferle [Thu, 18 Oct 2018 13:42:20 +0000 (15:42 +0200)] 
HISTORY: bug fixes and improvements of v1.1

Add history of bug fixes and improvements of v1.1
release.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks agoREADME: Top-level features
Jonas Pfefferle [Fri, 7 Sep 2018 14:32:28 +0000 (16:32 +0200)] 
README: Top-level features

Keep list of top level features. Change Contact -> Community.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks agoREADME: remove content, add disclaimer
Jonas Pfefferle [Thu, 6 Sep 2018 11:20:08 +0000 (13:20 +0200)] 
README: remove content, add disclaimer

Remove content which is provided in readthedocs to avoid
keeping track of multiple location when updating the documentation.
Add a disclaimer that we are a incubator project.
Add badges for license and docs build status.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-36
https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-24

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks ago[maven-release-plugin] prepare for next development iteration
Jonas Pfefferle [Thu, 18 Oct 2018 13:05:35 +0000 (15:05 +0200)] 
[maven-release-plugin] prepare for next development iteration

4 weeks ago[maven-release-plugin] prepare release v1.1-rc0 v1.1-rc0
Jonas Pfefferle [Thu, 18 Oct 2018 13:04:09 +0000 (15:04 +0200)] 
[maven-release-plugin] prepare release v1.1-rc0

4 weeks agoFix checkstyle xml header
Jonas Pfefferle [Thu, 18 Oct 2018 12:14:50 +0000 (14:14 +0200)] 
Fix checkstyle xml header

Move xml version to the top of the file to avoid illformed xml file.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-53

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks agoPOM: use tag HEAD in scm configuration
Jonas Pfefferle [Thu, 18 Oct 2018 12:10:24 +0000 (14:10 +0200)] 
POM: use tag HEAD in scm configuration

Use tag HEAD in scm configuration to avoid version increments by
release plugin for rc candidates.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks agoGenerate source tar files instead of zip
Jonas Pfefferle [Thu, 18 Oct 2018 11:20:03 +0000 (13:20 +0200)] 
Generate source tar files instead of zip

Generate source tar.gz files instead of zip files to be
consistent with the binary release.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-54

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 weeks agoAdd missing license headers
Jonas Pfefferle [Thu, 18 Oct 2018 10:45:51 +0000 (12:45 +0200)] 
Add missing license headers

Add missing license headers to docker files and exclude CREDITS file
from rat check.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-53

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 weeks agoThis commit fixes two minor issues for the docker build.
Adrian Schuepbach [Wed, 26 Sep 2018 14:11:34 +0000 (14:11 +0000)] 
This commit fixes two minor issues for the docker build.

The first issue is that core-site.xml can be 0 length after
substituting variables, see:
https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-51

The second issue is that CRAIL_HOME points to a hardcoded version, see:
https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-52

To fix these issues, the commit makes the following changes:

- Read Crail version from toplevel pom.xml
- Move Crail build directory to /home/crail
- Point CRAIL_HOME to /home/crail
- Use template core-site.xml.env file and subsitute environment variables
  to a new core-site.xml file

This closes #40

Signed-off-by: Adrian Schuepbach <asq@apache.org>
2 months agoCheckstyle: rename style file
Jonas Pfefferle [Fri, 7 Sep 2018 14:06:37 +0000 (16:06 +0200)] 
Checkstyle: rename style file

Rename file style.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoCheckstyle plugin
Jonas Pfefferle [Fri, 7 Sep 2018 12:49:22 +0000 (14:49 +0200)] 
Checkstyle plugin

Move checkstyle plugin to parent pom.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoCheckstyle plugin with google style
Jonas Pfefferle [Wed, 28 Mar 2018 20:54:19 +0000 (22:54 +0200)] 
Checkstyle plugin with google style

Add checkstyle plugin but do not check on compilation yet.
Checkstyle can be run with 'mvn checkstyle:check'

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-19

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoApache parent pom
Jonas Pfefferle [Tue, 21 Aug 2018 09:36:53 +0000 (11:36 +0200)] 
Apache parent pom

Use Apache parent pom. It already includes all necessary pieces
to do source and binary releases and push to maven central.

https://jira.apache.org/jira/projects/CRAIL/issues/CRAIL-34

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoDocumentation: remove logo
Jonas Pfefferle [Tue, 4 Sep 2018 09:01:56 +0000 (11:01 +0200)] 
Documentation: remove logo

Remove Crail logo from documentation.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoDocumentation: Spark-IO
Jonas Pfefferle [Wed, 15 Aug 2018 09:15:36 +0000 (11:15 +0200)] 
Documentation: Spark-IO

Add Spark-IO documentation: building and configuration.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoDocumentation: HDFS Adapter
Jonas Pfefferle [Wed, 15 Aug 2018 08:45:54 +0000 (10:45 +0200)] 
Documentation: HDFS Adapter

Add Spark HDFS Adapater documentation.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoDocumentation: iobench & fsck
Jonas Pfefferle [Tue, 14 Aug 2018 09:26:08 +0000 (11:26 +0200)] 
Documentation: iobench & fsck

Add iobench and fsck documentation.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoDocumentation: readthedocs Documentation
Jonas Pfefferle [Fri, 10 Aug 2018 08:25:01 +0000 (10:25 +0200)] 
Documentation: readthedocs Documentation

New Documentation using readthedocs based on github README.
Extended and adapted to latest Crail version.
Will be hosted at http://incubator-crail.readthedocs.io

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-24

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
2 months agoAdjusting to latest DiSNI/DaRPC (v1.7)
Patrick Stuedi [Tue, 4 Sep 2018 14:32:54 +0000 (16:32 +0200)] 
Adjusting to latest DiSNI/DaRPC (v1.7)

https://issues.apache.org/jira/browse/CRAIL-50

Signed-off-by: Patrick Stuedi <pstuedi@apache.com>
3 months agoDocker: fix cache hugepage requirement
Jonas Pfefferle [Fri, 3 Aug 2018 08:36:57 +0000 (10:36 +0200)] 
Docker: fix cache hugepage requirement

Allow users to use the Crail shell without providing
hugepages via a volume mount to Docker.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
3 months agoDocker build: TCP and RDMA tier
Jonas Pfefferle [Thu, 26 Jul 2018 11:35:25 +0000 (13:35 +0200)] 
Docker build: TCP and RDMA tier

Docker build files for the TCP and RDMA tier.
User can extend configuration to use NVMf or other tiers.

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-49

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 months agoNVMf: new jNVMf version 1.4
Jonas Pfefferle [Thu, 5 Jul 2018 11:43:52 +0000 (13:43 +0200)] 
NVMf: new jNVMf version 1.4

Use new jNVMf to fix dependency issue with DiSNI 1.6.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 months agoThrow more meaningful exception if datapath is invalid or has no write
Patrick Stuedi [Tue, 26 Jun 2018 08:43:04 +0000 (10:43 +0200)] 
Throw more meaningful exception if datapath is invalid or has no write
permissions.

Fixes the issue of invalid datapath or missing write permissions on the
datapath for both the RDMA and the TCP storage tier.

https://issues.apache.org/jira/browse/CRAIL-45

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
4 months agoMake sure statistics are updated across all classes in the Crail HDFS
Patrick Stuedi [Mon, 25 Jun 2018 11:46:45 +0000 (13:46 +0200)] 
Make sure statistics are updated across all classes in the Crail HDFS
module

Previously statistics where only captured in HDFSOutputStream and parts
of the CrailHadoopFileSystem.

https://issues.apache.org/jira/browse/CRAIL-26

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
4 months agoDelegate implementation of CrailHDFS to CrailHadoopFileSystem
Patrick Stuedi [Mon, 25 Jun 2018 11:16:01 +0000 (13:16 +0200)] 
Delegate implementation of CrailHDFS to CrailHadoopFileSystem

Implement AbstractFileSystem by delegating function calls to
CrailHadoopFileSystm. The same can be done also by using
DelegateToFileSystem from hadoop-common, but this results in additional
operations (e.g., exists(path) and checkPath(path)).

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-27

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
4 months agoAdding extra config parameters to control backlog and connnection
Patrick Stuedi [Fri, 22 Jun 2018 09:37:51 +0000 (11:37 +0200)] 
Adding extra config parameters to control backlog and connnection
timeouts for RDMA-based RPC and storage backends

The backlog parameter controlls the backlog of incoming connection
requests for rdma storage and rdma RPC. The connectiontimeout puts a
limit on the time to wait for the connection establishement event.

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
4 months agoAdjustments to comply with DiSNI v1.6 and DaRPC v1.6
Patrick Stuedi [Thu, 21 Jun 2018 14:40:26 +0000 (16:40 +0200)] 
Adjustments to comply with DiSNI v1.6 and DaRPC v1.6

The connect/bind API of DiSNI version 1.6 has changed, affecting
rpc-darpc and storage-rdma.

https://issues.apache.org/jira/browse/CRAIL-41

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
4 months agoScripts: fix log/pid file for multiple datanodes
Jonas Pfefferle [Wed, 20 Jun 2018 13:45:47 +0000 (15:45 +0200)] 
Scripts: fix log/pid file for multiple datanodes

Fix using the same filename when starting multiple storage
tiers on the same node. Append arguments from slave file
to make path unique.

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-29

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 months agoScripts: add crail-env.sh
Jonas Pfefferle [Wed, 20 Jun 2018 14:52:46 +0000 (16:52 +0200)] 
Scripts: add crail-env.sh

Add crail-env.sh file to allow setting parameters for
the start scripts like extra java options.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 months agoStart script: remove heap size argument
Jonas Pfefferle [Wed, 20 Jun 2018 11:36:11 +0000 (13:36 +0200)] 
Start script: remove heap size argument

Remove Xmx and MaxDirectMemorySize argument from start script
since it does not allow to run Crail on machines with less
than 64G.

https://issues.apache.org/jira/browse/CRAIL-40

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
4 months agoCheck for already existing cache directory with the same ID
Patrick Stuedi [Wed, 20 Jun 2018 13:06:02 +0000 (15:06 +0200)] 
Check for already existing cache directory with the same ID

Throw an exception if a cache directory for the same ID
(reflecting a particular instance of CrailStore) already exists

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
4 months agoIntroducing StorageUtils as a common ground for storage utility
Patrick Stuedi [Wed, 20 Jun 2018 12:11:04 +0000 (14:11 +0200)] 
Introducing StorageUtils as a common ground for storage utility
functions

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
4 months agoMake default configuration easier to run
Patrick Stuedi [Tue, 19 Jun 2018 13:21:07 +0000 (15:21 +0200)] 
Make default configuration easier to run

Better default values for cachepath and tcp storage path. Throw an
exception if paths are invalid. Throw an exception if interface is
invalid. Trim down crail-site.conf.template to only include the
necessary parameters.

https://issues.apache.org/jira/browse/CRAIL-15

Signed-off-by: Patrick Stuedi <pstuedi@apache.org>
6 months agoMove release date in HISTORY v1.0
Jonas Pfefferle [Tue, 15 May 2018 11:04:26 +0000 (13:04 +0200)] 
Move release date in HISTORY

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoApache incubator disclaimer
Jonas Pfefferle [Mon, 14 May 2018 11:02:57 +0000 (13:02 +0200)] 
Apache incubator disclaimer

Add Aapache incubator disclaimer.

https://issues.apache.org/jira/browse/CRAIL-33

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoMove IBM copyright NOTICE to CREDITS file
Jonas Pfefferle [Mon, 14 May 2018 10:59:28 +0000 (12:59 +0200)] 
Move IBM copyright NOTICE to CREDITS file

Move the IBM copyright notice from the NOTICE file to the CREDITS file.

https://issues.apache.org/jira/browse/CRAIL-33

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoNVMf: decrease keep alive interval
Jonas Pfefferle [Thu, 3 May 2018 11:37:44 +0000 (13:37 +0200)] 
NVMf: decrease keep alive interval

Set keep alive interval to lower value to not run into the
situation where we missed the keep alive deadline and the connection
is shutdown by the controller.

https://issues.apache.org/jira/browse/CRAIL-32

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoNVMf: fix staging buffer cache get race condition
Jonas Pfefferle [Mon, 7 May 2018 07:16:22 +0000 (09:16 +0200)] 
NVMf: fix staging buffer cache get race condition

Fixes NVMf staging buffer cache get. Do not throw an exception
if we can't free the previous buffer in the hashmap. If we can't
free it it was already freed by allocateFreeBuffers.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoNVMf: staging buffer cache clear buffer
Jonas Pfefferle [Thu, 3 May 2018 11:38:00 +0000 (13:38 +0200)] 
NVMf: staging buffer cache clear buffer

Clear buffer position and limit when getting the buffer of the
free queue since position and limit might have been modified before.

https://issues.apache.org/jira/browse/CRAIL-31

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoNVMf: staging buffer cache existing entry bug
Jonas Pfefferle [Wed, 2 May 2018 10:53:47 +0000 (12:53 +0200)] 
NVMf: staging buffer cache existing entry bug

A buffer might already exist at a certain remote address (resp. LBA) if
a file was deleted and a block was reassigned to a different file. Do not
throw an exception but free the buffer. If we can't free it something
went wrong as there should never be two writers writing to the same
remote address.

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-30

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoREADME: give all three options to download src
Jonas Pfefferle [Mon, 23 Apr 2018 10:54:26 +0000 (12:54 +0200)] 
README: give all three options to download src

Include all three options to download the Crail source:
Apache, github and Apache source distro.

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-24

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoMove HISTORY release date
Jonas Pfefferle [Mon, 23 Apr 2018 09:21:00 +0000 (11:21 +0200)] 
Move HISTORY release date

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoREADME: build from source distro
Jonas Pfefferle [Mon, 23 Apr 2018 09:19:27 +0000 (11:19 +0200)] 
README: build from source distro

Add url for source distro and build instructions.

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-24

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
6 months agoCopyright NOTICE: change to use actual year
Jonas Pfefferle [Mon, 23 Apr 2018 09:11:41 +0000 (11:11 +0200)] 
Copyright NOTICE: change to use actual year

Change copyright notice for the Apache Software Foundation
to use the actual year instead of "onwards".

https://issues.apache.org/jira/browse/CRAIL-28

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoMove release date in HISTORY
Jonas Pfefferle [Wed, 18 Apr 2018 07:45:15 +0000 (09:45 +0200)] 
Move release date in HISTORY

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoAdd license header to md files
Jonas Pfefferle [Mon, 16 Apr 2018 15:37:16 +0000 (17:37 +0200)] 
Add license header to md files

Add license header to md files and remove exclude from rat.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoMove copyright notice to NOTICE file.
Jonas Pfefferle [Mon, 16 Apr 2018 15:31:10 +0000 (17:31 +0200)] 
Move copyright notice to NOTICE file.

Remove copyright notice from license headers and move it
to NOTICE file.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoFix LICENSE file
Jonas Pfefferle [Fri, 13 Apr 2018 08:56:04 +0000 (10:56 +0200)] 
Fix LICENSE file

LICENSE file should only include licenses that we use.
Now only includes Apache License 2.0

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoCheck licenses with apache-rat
Jonas Pfefferle [Fri, 13 Apr 2018 08:53:59 +0000 (10:53 +0200)] 
Check licenses with apache-rat

Add apache-rat maven plugin to allow to check licenses with
mvn apache-rat:check

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoAdd missing license headers
Jonas Pfefferle [Fri, 13 Apr 2018 08:52:48 +0000 (10:52 +0200)] 
Add missing license headers

https://issues.apache.org/jira/browse/CRAIL-25

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agov1.0 release history
Jonas Pfefferle [Thu, 12 Apr 2018 08:58:13 +0000 (10:58 +0200)] 
v1.0 release history

Close #19

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoREADME: update NVMf tier constants
Jonas Pfefferle [Tue, 10 Apr 2018 13:06:19 +0000 (15:06 +0200)] 
README: update NVMf tier constants

Update NVMf tier constants in README.

https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-24

Close #18

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoNew jNVMf version 1.3
Jonas Pfefferle [Tue, 10 Apr 2018 12:12:05 +0000 (14:12 +0200)] 
New jNVMf version 1.3

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoNew DiSNI and DaRPC version
Jonas Pfefferle [Tue, 10 Apr 2018 12:03:32 +0000 (14:03 +0200)] 
New DiSNI and DaRPC version

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoCrail internal version to 3100
Jonas Pfefferle [Tue, 10 Apr 2018 11:08:59 +0000 (13:08 +0200)] 
Crail internal version to 3100

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>
7 months agoNvmfUnalignedWriteFuture: refactoring
Jonas Pfefferle [Tue, 10 Apr 2018 09:08:50 +0000 (11:08 +0200)] 
NvmfUnalignedWriteFuture: refactoring

Use offsetInSector in floorToSector and provide a new
floorToSector overload method for ints.

Signed-off-by: Jonas Pfefferle <pepperjo@apache.org>