Original Posting:
>
> Can anyone give me some tips, or point me to locations where
> I can learn about backing up clearcase. I've been given the
> task of doing this and it appears to be a bit tricky.
Thanks to:
Paul Kanz <paul@icx.com>
Robert Jennings <rjenning@informix.com>
Richard C. Mills" <rmills@atl1.america.net>
seanm@sybase.com (Sean McInerney)
Manish Doshi <mdoshi@guanine.bchs.uh.edu>
poffen@San-Jose.ate.slb.com (Russ Poffenberger)
All responders agreed that backing up the VOB was of utmost
importance. This is accomplished by locking the VOB, and
then archiving with your tool of choice. There was some
disagreement about the importance of backing up the view.
My investigation along with the responses I received convinced
me that backing up the views is extremely important.
Thoughts on Views:
When you look at a view, it appears to be a directory structure
consisting of your development environment,,,sources, build targest,
etc. Much of this is what I call "virtual baloney", it is constructed
from the VOB, and the way your view is defined. Now, when you check
out a source file, or create a new file, this is totally outside
the VOB, and will not be preserved by backing up the VOB until it
is checked back in. So,,,,just where do these files actually live?
If you do a cleartool lsview you get something like:
swt /user/swt/views/swt.vws
The directory swt.vws, contains these real files. This is what
I have decided to back up view-wise.
Thanks to all.
For the sake of the responders, I will include their responses in
full. Additionally, I contacted them directly with further questions
and will also include those responses.
And last but not least,,,when I actually write my backup scripts
and get them working I will forward them to the responders.
-Mark
Mark-
-----------------------------------------------------------------------
Paul Kanz <paul@icx.com>
It's a piece of cake! Don't bother backing up the Views, that data
comes
from the VOB and the amount of time it takes to restore part of a
View...
But, you MUST backup the VOB! There are two methods you can backup a VOB
-
to disk or tape.
To disk (I'm not doing this currently, but I will when I get a spare
disk):
1. Lock the VOB(s)
cleartool lock -vob /<vob_path>/<vob_name>.vbs
2. Backup the VOB
tar -cf /<path>/<vob_name>.vbs.tar
3. Unlock the VOB(s)
cleartool unlock -vob /<vob_path>/<vob_name>.vbs
4. Then go back and gzip or compress the tar file
gzip -9 /<path>/<vob_name>.vbs.tar
5. Then backup this drive to tape.
This will give you the least amount of time that the VOB needs to be
locked. Depending how much space is on the disk, I would keep several
copies of each VOB.
To tape (this is what I'm doing now):
#!/bin/sh
# atria_maint.sh - ran from cron on VOB server; performs weekend
maint. This
# should be ran from cron, as vobadm.
# crontab entry:
# 59 21 * * 6 /users/vobadm/atria_maint.sh
# 6th day of the week (Sat) at 9:59pm
# Paul Kanz 03/07/96
# LAST MODIFY: 03/07/96
#
CLEARCASE_PATH=/nfs/atria/usr/ccase_rls/clearcase_v2.1/sun4/sun4-4.n/bin
PATH=/bin:/usr/ucb:/etc
#
$CLEARCASE_PATH/cleartool lock -vob /<path>/vbs/<vob_name>.vbs
#
tar -cvbf 60 - /usr2/vbs | rsh tapehost dd of=/dev/rmt/0m obs=60b;
#
$CLEARCASE_PATH/cleartool unlock -vob /<path>/vbs/<vob_name>.vbs
rsh tapehost mt -t /dev/rmt/0m offl
#
#EOF
-Paul
PS - I wouldn't mind seeing your backup/restore scripts.
-----------------------------------------------------------------------
seanm@sybase.com (Sean McInerney)
I use the "/usr/atria/bin/cleartool lock -vob /foodir/foo.vbs" command
before I backup vobs and the cleartool unlock when I'm done.
I use ufsdump (level 0 only) on 2.x. Views are on individual
workstations or departmental servers and are level 0 dumped
each night along with the vobs.
....Sean
-----------------------------------------------------------------------
Robert Jennings <rjenning@informix.com>
Just make sure you lock the vobs before you back them up. Once they are
locked you can treat them as normal file system backups. Use the command
$ATRIAHOME/bin/cleartool lock -vob {vobname}
where vobname is the location of the vob e.g. /cm/vobs/xxx.vob
After backup you just unlock the vobs. Replace the word "lock" with
"unlock" to unlock them.
We just treat the views as normal filesystem backups if a user is using
it while it is being backed up then the restore will reflect this. This
is generally not a problem as we backup at night.
-----------------------------------------------------------------------
Richard C. Mills" <rmills@atl1.america.net>
Here are a few scripts I got from a friend that his company uses to
backup
their VOBS (remember, the VOBS have to be locked before you back them
up).
Enjoy!
BTW, their clearcase was run on a mix of AIX/Solaris.
#!/bin/ksh
# copy_vobs.sh - C. Selden, 5/23/96 - Locks, tars and then unlocks *.vbs
# directories (VOBS) to central repositories (/vobsaveX).
# Version 1.1
# - 6/7/96 C. Selden -
# Added condition to du - only done on cm2.
#########################################################################
# Version 1.0
# Initial Release
##########################################################################
HOST=`uname -n`
DATE=`date +%m%d`
OUTFILE="/sysadm/copy_vob_logs/copy_vob.out.$HOST.$DATE"
ERRFILE="/sysadm/copy_vob_logs/copy_vob.err.$HOST.$DATE"
NOTIFY="cselden@rs2b"
# Remove any left-over log files.
rm -f $OUTFILE > /dev/null 2>&1
rm -f $ERRFILE > /dev/null 2>&1
# Create NFS mount to cm2:/vobsave1, if not on cm2
if [ "$HOST" != "cm2" ]
then
/usr/etc/mknfsmnt -f '/vobsave1' -d '/vobsave1' -h 'cm2' -m 'NFS' '-n'
'-B' '-A' -t 'rw' -w 'bg' '-S' '-Y' '-Z' '-X' 2>> $ERRFILE
fi
# List current contents of /vobsave
echo "Original contents of /vobsave1:" >> $OUTFILE
ls -l /vobsave1 >> $OUTFILE
echo "------------" >> $OUTFILE
if [ "$HOST" = "cm2" ]
then
echo "Total prior size of /vobsave1: " >> $OUTFILE
du -sk /vobsave1 >> $OUTFILE
fi
echo "------------" >> $OUTFILE
# Build array of /vobstoreX partitions
DIRS=`df | grep dev | grep vobstore | awk '{ print $7 }'`
# For each of these partitions, build an array of *.vbs directories to
# tar up.
for d in $DIRS
do
# Keep track of time for each partition, and each vob.
echo "Starting copy of $d at `date`" >> $OUTFILE
cd $d
VOBS=`ls | grep .vbs`
# make sure there are vobs to copy
if [ "$VOBS" != "" ]
then
for v in $VOBS
do
# make sure there are vobs to copy
# Lock each VOB, tar it to /vobsave1, then unlock it.
echo "Starting copy of $d/$v at `date`" >> $OUTFILE
/usr/atria/bin/cleartool lock -vob $v >> $OUTFILE 2>> $ERRFILE
tar cvf /vobsave1/$v.tar $v > /dev/null 2>> $ERRFILE
echo "Finished copy of $d/$v at `date`" >> $OUTFILE
/usr/atria/bin/cleartool unlock -vob $v >> $OUTFILE 2>> $ERRFILE
done
echo "Finished copy of $d at `date`" >> $OUTFILE
else
echo "No VOBs in $d. Skipping..." >> $OUTFILE
fi
cd /
done
# Show the updated contents of /vobsave1
echo "------------" >> $OUTFILE
echo "Updated contents of /vobsave1:"
ls -l /vobsave1 >> $OUTFILE
if [ "$HOST" = "cm2" ]
then
echo "Total new size of /vobsave1: " >> $OUTFILE
du -sk /vobsave1 >> $OUTFILE
fi
echo "---End of Report---------" >> $OUTFILE
# Remove NFS mount.
if [ "$HOST" != "cm2" ]
then
cd /
/usr/etc/rmnfsmnt -f '/vobsave1' '-B' 2>> $ERRFILE
fi
# Send out log files
if [ -s $ERRFILE ]
then
mailx -s "$ERRFILE" $NOTIFY < $ERRFILE
else
rm -f $ERRFILE > /dev/null 2>&1
fi
mailx -s "$OUTFILE" $NOTIFY < $OUTFILE
### End of vob copy
#!/bin/ksh
# check_licenses.sh - C. Selden, 5/30/96. Checks on license usage for
# ClearCase and ClearTrack products.
DATE=`date +%m%d`
LOGFILE="/sysadm/clearlogs/clearlic_log.$DATE"
LIC_CMD="/usr/atria/bin/clearlicense"
if [ ! -f $LOGFILE ]
then
echo "ClearCase and ClearTrack License Log for `date`" >> $LOGFILE
echo "---------------------------------------------------------" >>
$LOGFILE
fi
CC_LIC=`$LIC_CMD -product ClearCase | grep "Current active" | awk
'{print $4}'`
if [ "$CC_LIC" != "0" ] && [ "$CC_LIC" != "" ]
then
echo "`date`: ClearCase Licenses in Use: $CC_LIC" >> $LOGFILE
echo "---" >> $LOGFILE
fi
CT_LIC=`$LIC_CMD -product ClearTrack_C | grep "Current active" | awk
'{print $4}'`
if [ "$CT_LIC" != "0" ] && [ "$CT_LIC" != "" ]
then
echo "`date`: ClearTrack Licenses in Use: $CT_LIC" >> $LOGFILE
echo "---" >> $LOGFILE
fi
### ENd of check license
#!/bin/ksh
# mail_clearlog.sh - C. Selden 5/30/96. Mails Atria product license
# logs.
DATE=`date +%m%d`
LOGFILE="/sysadm/clearlogs/clearlic_log.$DATE"
NOTIFY="cselden@rs2b"
if [ -f $LOGFILE ]
then
mailx -s "ClearCase/ClearTrack License Usage" $NOTIFY < $LOGFILE
fi
#### ENd of mail logs.
----------------------------------------------------------------------
From:
"Richard C. Mills" <rmills@atl1.america.net>
To:
mjb@ltx.com (Mark Belanger)
> If userX is working in his view, and either checks out a source file
> and/or creates a new source file that he intends to check in as a
> new module, will backing up only the VOB preserve this data if
> userX loses or removes the checked-out source file and new source file?
Nope. You'll need to back up their view as well
> It seems to me that in addition to backing up the VOBS, I should
> also backup every individual view. In each individual view, I should
> determine which files are checked out or new files, and back them
> up. I don't think it is feasible to backup the entire view, as our
> development environment is huge and the vast majority of a view
> is virtual baloney along with real files (those that are checked out
> or new).
Well, we bought a jukebox along with legato just for this purpose. It
is quite a bit of data (40 GB for us)
> One last question, and pardon me for my ignorance. If you go into
> a view, and create a new file, does this file actually exist on
> some filesystem somewhere? For instance, if I create a file in
> oasis:/view/mjb is it a real file somewhere, or does clearcase
> somehow keep track of it in the vob database?
I believe clearcase keeps track of it.
> Thanks again for your assistance, my grandchildren will sing your
> praises around campfires for generations to come.
Allright! And I thought I'd have to do something of significance. -_^
Rich
----------------------------------------------------------------------
Subject:
Re: thanks - more ccase info?
Date:
Wed, 25 Sep 1996 11:44:51 -0700 (PDT)
From:
Paul Kanz <paul@icx.com>
To:
Mark Belanger <mjb@ltx.com>
CC:
seanm@sybase.com, jenning@informix.com, rmills@atl1.america.net
Mark-
See my response below. I'd like to see what the other Atria users
response/thoughts are.
On Wed, 25 Sep 1996, Mark Belanger wrote:
> If userX is working in his view, and either checks out a source file
> and/or creates a new source file that he intends to check in as a
> new module, will backing up only the VOB preserve this data if
> userX loses or removes the checked-out source file and new source file?
No. If a user checks out source from the VOB, it is stored in a View
(most likely their personal View). Until the source is checked back
into
the VOB, the VOB will not contain the users mods.
> It seems to me that in addition to backing up the VOBS, I should
> also backup every individual view. In each individual view, I should
> determine which files are checked out or new files, and back them
> up. I don't think it is feasible to backup the entire view, as our
> development environment is huge and the vast majority of a view
> is virtual baloney along with real files (those that are checked out
> or new).
Backing up individual views are a grey area; I'd like to hear what
others
think about this and/or how they are handling this. Currently I do not
backup the 'main' views, I do backup the users views in $HOME.
> One last question, and pardon me for my ignorance. If you go into
> a view, and create a new file, does this file actually exist on
> some filesystem somewhere? For instance, if I create a file in
> oasis:/view/mjb is it a real file somewhere, or does clearcase
> somehow keep track of it in the vob database?
Keep in mind that Atria is it's own file system (MVFS), which is a
replacement for tfs; it 'sits on top' of NFS. You SHOULD NOT use unix
commands (rm, mv, cp, etc...) inside of /view. Always use the ClearCase
commands or be at a ClearCase prompt when doing any sort of file
manipulation.
>From a ClearCase point of view, its a real file - from a unix point of
view, its a long file name that does not make much sense. This
file/name
translation is handled by the registry.
The registry is one issue that makes restoring a View VERY difficult; in
some cases (read most) its best to unregister the View, restore it from
tape and then register it. Trying to restore one file is next to
impossible.
Version 3 is suppose to resolve some of these restore issues.
-Paul
----------------------------------------------------------------------
Subject:
Re: thanks - more ccase info?
Date:
Thu, 26 Sep 1996 10:43:54 -0400
From:
seanm@sybase.com (Sean McInerney)
To:
mjb@ltx.com, paul@icx.com
CC:
jenning@informix.com, rmills@atl1.america.net, seanm@sybase.com
> From paul@icx.com Wed Sep 25 15:00 EDT 1996
> Return-Path: <paul@icx.com>
>
> Mark-
>
> See my response below. I'd like to see what the other Atria users
> response/thoughts are.
>
> On Wed, 25 Sep 1996, Mark Belanger wrote:
>
> > If userX is working in his view, and either checks out a source file
> > and/or creates a new source file that he intends to check in as a
> > new module, will backing up only the VOB preserve this data if
> > userX loses or removes the checked-out source file and new source file?
>
> No. If a user checks out source from the VOB, it is stored in a View
> (most likely their personal View). Until the source is checked back into
> the VOB, the VOB will not contain the users mods.
>
True for us. Each workstation has a local user disk which
is $HOME (15-18MB debug images don't make NFS mounted home dirs
very practical here. 100-200MB debug on AIX and HP/UX!). A
developer may have code checked out into his/her view for days...
So of course we have to back up views. Several times I have restored
the users "view" (to someplace) so they can get source back.
The main VOB here is about 9GB and I nightly, level 0 dump,
about 20GB of $HOME and other related views (7 days of nighly build
views, etc..., 6 different platform branches).
Some Ccase shops have VOB servers, View servers and Build
servers
but we distribute the views to each workstation. The downside is
the tremendous ethernet traffic we generate when someone touches
the world (some .h file change for instance) which may cause lots
of new objects to be moving around the net, and at night when
we "rmt" them back over to tape. 100BT will become mandatory
here soon, not just "nice" to have.
> > It seems to me that in addition to backing up the VOBS, I should
> > also backup every individual view. In each individual view, I should
> > determine which files are checked out or new files, and back them
> > up. I don't think it is feasible to backup the entire view, as our
> > development environment is huge and the vast majority of a view
> > is virtual baloney along with real files (those that are checked out
> > or new).
>
> Backing up individual views are a grey area; I'd like to hear what others
> think about this and/or how they are handling this. Currently I do not
> backup the 'main' views, I do backup the users views in $HOME.
>
> The registry is one issue that makes restoring a View VERY difficult; in
> some cases (read most) its best to unregister the View, restore it from
> tape and then register it. Trying to restore one file is next to
> impossible.
We have done the impossible many times.... People make
mistakes and backups are for correcting that as well. You can
put back "the one" file.
Just my introductory thoughts on how an SA deals with
his Ccase environment.
....Seanm
----------------------------------------------------------------------
Subject:
Re: backing up clearcase
Date:
Thu, 26 Sep 96 08:37:41 PDT
From:
poffen@San-Jose.ate.slb.com (Russ Poffenberger)
To:
mjb@ltx.com
Clearcase VOB's and views are just filesystem based databases. All you
need to
do is backup the vob or view storage directory. The trick is that the
VOB or
view must be idle (ie, no changes) during the course of the backup, or
the
backup is useless because the database is inconsistent over a period of
time.
VOB's are the most critical, so you must lock the VOB to make sure that
no
changes occur during the backup. The downside is that no operations can
be
performed by the users that would change the VOB (checkin, checkout,
compiles
because of derived object mechanisms, etc) while it is locked. Our build
scripts and makefile's can detect when a VOB is locked and sleep until
it
is unlocked. Our backup mechanism does a quick copy of the VOB storage
to
another disk (which is faster, so the VOB is locked for a shorter period
of
time), then the backup is done to tape off the copy.
We spent a lot of time developing this. The good thing (especially for
you)
is that the upcoming clearcase 3.0 has built a lot of this in, a VOB
replica
mechanism for backing up, automatic sleeps when a VOB is locked, etc.
----------------------------------------------------------------------
Subject:
Re: backing up clearcase
Date:
Wed, 25 Sep 1996 14:29:52 -0500 (CDT)
From:
Manish Doshi <mdoshi@guanine.bchs.uh.edu>
To:
mjb@ltx.com
Hi,
If you do ufsdump (or dump on SunOS) on the device /dev/dsk/c#...
you wil be fine. Vobs and views are no diff. than regular UNIX file
systems.
VOBS (Versioned Object Base) sits on top of NFS. and views have
theri own view private dir undeneath they have some subdirs for
view private files.
If you have disk space problem or back up problem then
you don't want to back up the c subdir as it's just a cache(ClearText)
dir. But make sure you lock the VOB before you do back up.
You have to be a root to lock the VOB. I'm not sure if
VOB owner can lock the VOB or not.
----------------------------------------------------------------------
-- Mark Belanger Email : mjb@ltx.com LTX Corporation Fax : (617) 329-6880 LTX Park at University Ave Voice : (617) 467-5021 Westwood Ma 02090 http://www.tiac.net/users/mjb
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:11:11 CDT