StorWIS Research Big Data Storage

Minimum Client Requirements for Connection to CIFS/SMB

The clients listed in the following table are certified to work with StorWIS.

Usage Instructions

To access a lab StorWIS folder, you must first register with Internal Services for the StorWIS - Big Data Storage service, then map StorWIS to the network drive, as appropriate for the particular platform you’re using (Windows, Mac and Linux), as detailed below.

Windows

On Windows PCs, the network drive needs be created and mapped manually. To do so:

1. Open My Computer (or Computer, depending on the version of Windows being used).
2. Click Map Network Drive.
3. In the dialog that opens, enter the complete folder path (i.e. \\isi.storwis.weizmann.ac.il\labs\lab_name) in the Folder field, check Reconnect at sign-in, check Connect using different credentials and click Finish.
   
4. When mapping the drive, enter your user ID and password when prompted to do so (be sure to enter wismain\ right before your user ID as shown below), and check Remember my credentials.
   

To restore files from system snapshots:

1. Right-click the desired folder, click Properties in the context menu that pops up, then click the Previous Versions tab in the dialog that opens.
   
2. Locate the file or folder you wish to restore and click Copy… (so as to prevent current files from being overwritten) or Restore… (potentially overwriting current files).

You may alternatively restore files from snapshots by navigating to the hidden .snapshot folder located within each folder and copying any file you wish to restore from that folder to a desired destination.

Mac

To connect to the service from a Mac:

1. Select the Connect to Server command from the Apple menu or press Cmd + K from Mac OS Finder. The following window appears.
   
2. When connecting via SMB, enter the appropriate path, starting with smb:// in the Server Address field, as in the following example:
   smb://isi.storwis.weizmann.ac.il/labs/shalomlab.
   
   If connecting via CIFS, enter the appropriate path, starting with cifs:// in the Server Address field, as in the example below:
   cifs://isi.storwis.weizmann.ac.il/labs/shalomlab.
3. When mapping the drive, you will be prompted for a user name and password. Enter your account’s user ID with the wismain\ prefix and your password, and check Remember this password in my keychain.
   

Linux

To connect to the StorWIS service from Linux:

1. Run the following command:
   smbclient \\\\isi.storwis.weizmann.ac.il\\labs -U wismain/user_id (user ID in example below is aaext4).
   
2. Navigate to your lab directory via the following command:
   smb: \>cd lab_name
3. List the contents of the directory as follows:
   smb: \lab_name\> ls
   
   A listing similar to the following will be presented (the lab name in the example below is testlab).
   
4. To mount the StorWIS cifs volume in /etc/fstab, run the following command:
   //isi.storwis.weizmann.ac.il/labs/STORWIS cifs username=wismain\user_ID,password=password,defaults 0 0
5. The commands listed in the following table are available.
   
6. Enter the help command to list all smbclient commands (see below).
   
7. To receive help for individual commands, type ? followed by the name of the command (see samples below).
   
8. To capture the output of local commands, precede their name with ! (see examples below).
   
9. To download all files matching a desired mask, enter the following command:
   mget [mask] (mask in following example is dir1).
   
10. To disable confirmation prompts, enter the prompt command prior to the mget command (see example below).
    

Minimum Client Requirements for Connection to CIFS/SMB

The clients listed in the following table are certified to work with StorWIS.

Usage Instructions

To access a private user StorWIS folder, you must first register with Internal Services for the StorWIS - Big Data Storage service, then map StorWIS to the network drive, as appropriate for the particular platform you’re using (Windows, Mac and Linux), as detailed below.

Windows

On Windows PCs, the network drive needs be created and mapped manually. To do so:

1. Open My Computer (or Computer, depending on the version of Windows being used).
2. Click Map Network Drive.
3. In the dialog that opens, enter the complete folder path (i.e. \\isi.storwis.weizmann.ac.il\users\user_ID) in the Folder field, check Reconnect at sign-in, check Connect using different credentials and click Finish.
   
4. When mapping the drive, you will be asked to enter your user ID and password. Do so when prompted (be sure to enter wismain\ right before your user ID as shown below), and check Remember my credentials.
   

To restore files from system snapshots:

1. Right-click the desired folder, click Properties in the context menu that pops up, then click the Previous Versions tab in the dialog that opens.
   
2. Locate the file or folder you wish to restore and click Copy… (so as to prevent current files from being overwritten) or Restore… (potentially overwriting current files).

You may alternatively restore files from snapshots by navigating to the hidden .snapshot folder located within each folder and copying any file you wish to restore from that folder to a desired destination.

Mac

To connect to the service from a Mac:

1. Select the Connect to Server command from the Apple menu or press Cmd + K from Mac OS Finder. The following window appears:
   
2. When connecting via SMB, enter the appropriate path, starting with smb:// in the Server Address field, as in the following example:
   smb://isi.storwis.weizmann.ac.il/users/shalomg.
   
   If connecting via CIFS, enter the appropriate path, starting with cifs:// in the Server Address field, as in the example below:
   cifs://isi.storwis.weizmann.ac.il/users/shalomg.
3. When mapping the drive, you will be prompted for a user name and password. Enter your account’s user ID with the wismain\ prefix and your password, and check Remember this password in my keychain.
   

Linux

To connect to the StorWIS service from Linux:

1. Run the following command:
   smbclient \\\\isi.storwis.weizmann.ac.il\\Users -U wismain/user_id (user ID in example below is aaext4).
   
2. Navigate to your user directory via the following command:
   smb: \>cd user_id
3. List the contents of the directory as follows:
   smb: \user_id\> ls (user ID in example below is aaext4).
   
   A listing similar to the following will be presented.
   
4. To mount the StorWIS cifs volume in /etc/fstab, run the following command:
   //isi.storwis.weizmann.ac.il/Users /STORWIS cifs username=wismain\user_ID,password=password,defaults 0 0
5. The commands listed in the following table are available.
   
6. Enter the help command to list all smbclient commands (see below).
   
7. To receive help for individual commands, type ? followed by the name of the command (see samples below).
   
8. To capture the output of local commands, precede their name with ! (see examples below).
   
9. To download all files matching a desired mask, enter the following command:
   mget [mask] (mask in following example is dir1).
   
10. To disable confirmation prompts, enter the prompt command prior to the mget command (see example below).
    

This quick start provides guidance on the steps required to set up your StorWIS for use.

If you wish to store results or Office data, it is recommended that you use TBA Box services TBA.

Users from the following authorized units and departments may register for the StorWIS service:

• Faculties.
• Departments associated with faculties.
• Auxiliary units associated with faculties (registration is to be performed solely via the faculty PI).
• Science instruction units.

Minimum Client Requirements for Connection to CIFS/SMB

The clients listed in the following table are certified to work with StorWIS.

Registration to the StorWIS Lab Folder Service

Registration is to be performed by your faculty administrator via Internal Services.

Submitting a New Order

StorWIS Lab Folder Registration Procedure for PIs

Registration for PIs is to be performed as follows:

1. Enter your PI User ID and register the StorWIS Lab Folder service via Internal Services.
2. Specify the disk size (in GBs) that you wish to allocate (100 GBs minimum).
   
3. Assign a convenient name to your parent lab directory. For example, if the PI’s name is John Doe, the folder name can be JohndoeLab, JohnDLab, DJohnLab or similar.
   
4. Select a budget user number.
   
5. Click Send.
6. At this time, the PI status will appear as Registering.
   
7. Once the lab folder has been created (this may take a few hours), the PI status will change to Registered.
   
8. Upon completion of PI registration (with the PI status having changed to Registered), a new folder will be created at the following path:
   \\isi.storwis.weizmann.ac.il\labs\Lab-group-name\user-ID.

Submitting an Additional Order

Registration to the StorWIS Lab Member Folder Service

Registration Procedure for Lab Members

As lab member, your PI can add you to his or her StorWIS lab folder.

Registration for the “StorWIS Lab Member Folder” is to be performed as follows:

1. In Internal Services, enter your User ID and click Register next to the StorWIS Lab Member folder option.
   
2. To add a lab member to the lab’s folder, select your PI group name.
   
3. Choose your PI from the list of PIs that appears.
   
4. The data owner (the PI) will receive an e-mail message asking to approve the request to join his or her lab (see sample message below). To do so, the PI must click Click here to approve, following which the member will gain access to the lab folder.

Dear Dr. John Doe,

The user NMIHALPT belonging to Fred Wills has requested to use your group resources as StorWIS lab group: Johndoe.

Click here to approve.

Best regards,
Internal services

Please contact the HelpDesk at 4444 for any further clarification or assistance.

5. Click Send.
6. Wait an hour for the service lab folder member to be created (during this time, the lab member’s status will appear as Registering).

 Once the lab folder member has been created, the status will change to Registered.

8. Upon registration completion (and the lab member status having changed to Registered), a new folder will be created at the following path:
   \\isi.storwis.weizmann.ac.il\labs\Lab-group-name\user-ID.

StorWIS Private Folder Registration

Registration Procedure for PIs and Lab Members

PIs or lab members requiring a folder separate from the lab folder and solely accessible to them can register for a StorWIS private folder. Registration is to be performed as follows:

1. In Internal Services, enter your User ID and click Register next to the StorWIS Private folder option.
   
2. Specify the disk size (in GBs) and click Send.
   
3. Wait a few hours for the StorWIS private folder to be created (during this time, the PI or lab member’s status will appear as Registering).
   
4. Once the StorWIS ​private folder has been created, the status will change to Registered.
   
5. Upon registration completion (and the status having changed to Registered), a new folder will be created at the following path:
   \\isi.storwis.weizmann.ac.il\users\user-ID.

Mapping a Lab Folder and Lab Members to a Network Drive

Please refer to the following resources for information on how to map a lab or member folder to a network drive on computers running Windows, macOS or Linux:

• TBA Mapping a lab folder to a network drive TBA (link broken on old site).
• TBA Mapping lab members to a network drive TBA (link broken on old site).

Backup service is optional for StorWIS users. It may be enabled or disabled for a specific lab at any time via Internal Services.

Backup service charges cover daily backups with a 4-week retention period.

Files are backed up to two (on and off-site) copies and versions (for easy error recovery), and are retained a predetermined number of days, depending on specific applicable policy, as per the following table.

The retention policy for files stored on StorWIS is as follows:

• Number of versions stored: the number of backup versions that will be stored for a file that has been located on and not been erased from StorWIS.
• Retention time in days: the number of days that a backup version of a file will be retained. As the time in days passes, the very last copy of a specific file will be retained. In accordance with the duration of the currently selected retention policy (30 days), and the maximum number of versions allowed (10), versions of a file, the age of which exceeds 30 days, will be deleted.

Additionally, the retention policy for files that have been erased from StorWIS is as follows:

• Number of versions stored: the very latest version of a file that has been erased from StorWIS will be retained.
• Retention time in days: the number of days that the absolutely latest version of a file will be retained. When this time period in days has gone by, the file will be erased.
• Number of copies relates to the number of copies of a file stored on the Institute’s system. The value is 2, meaning that a single additional copy of the file will be stored outside the Institute, at a dedicated storage facility.

It is highly recommended that you exclude any files not necessarily having to be backed up (such as logs, temporary files, error files, etc.), as this can significantly reduce your backup charges. See the Registration for Backup Service section for detailed explanations on how to exclude files.

Registration for Backup Service

Registration for the StorWIS backup service is to be performed through your departmental administrator.

Request the service via Internal Services as follows:

1. On the following page, under List of Services, select Backup, then click the Go button.
   
2. Fill out the Debit userid, Application Owner and Email of person in charge of computer fields as appropriate when the following page loads, and click the Next button.
   
3. If registering for StorWIS lab backup, when the following page appears, use the Action control under Backup Details to select the STORWIS LAB BACKUP option. Be sure to exclude any files not necessarily having to be backed up (such as logs, temporary files, error files, etc.) using the appropriate file type and folder path exclusion controls at the bottom of the page, then click the Send button.
   
4. If registering for StorWIS private backup, when the following page appears, use the Action control under Backup Details to select the STORWIS PRIVATE BACKUP option. Be sure to exclude any files not necessarily having to be backed up (such as logs, temporary files, error files, etc.) using the appropriate file type and folder path exclusion controls at the bottom of the page, then click the Send button.
   

Service Charges

Backup charges may vary due to the data compression (de-duplication*) mechanisms employed. The general rule of thumb is that any repetitive data stored on StorWIS is not to be calculated more than once with regards to backup service charges.

See WIT charges for backup service charge baselines, and for further information on monthly rates.

* Data de-duplication is a specialized data compression technique that effectively eliminates redundant copies of data. Related and somewhat synonymous sub-file block items may not be accounted as new backup data.

The hardware employed to provide the StorWIS service is specifically designed for high availability. Redundancy is therefore provided for the following components:

• Storage array controllers.
• Power & network connectivity.
• Data on disk is also protected via various protection mechanisms, including hardware-level health checks, with support for automated 'call-home' to the manufacturer's support center.

All arrays are monitored with alarming for errors and any potential failure 24 hours a day, 7 days a week. This high level of monitoring, combined with redundancy built into the storage controller arrays, allows us to assure the highest possible solution availability.

Service Availability

The StorWIS storage service is operated by our IT staff to guarantee 99.9% availability, and is intended to remain accessible to all research groups at all times, except during scheduled maintenance windows and preventive maintenance updates.

Thanks to the service’s resilient design, outages are expected to be relatively rare, and will be limited to patches and upgrades requiring entire arrays to be shut down, and/or infrastructure (network, power) related interruptions. Any outages – whether planned or unscheduled – will be posted on our System Health and Bulletin Board, which is accessible via our site’s homepage.

Refer to the following table for details on various StorWIS service elements.

StorWIS Folder Structure

The following figure presents the StorWIS service’s folder structure.