Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between pages "FAQ How can I rearrange Eclipse views and editors?" and "EMFT Build Server Setup"

(Difference between pages)
 
m (more debug details)
 
Line 1: Line 1:
Eclipse allows you a great deal of flexibility in how the views and editors
+
= EMFT Build Server Setup =
are presented in the workbench.  With the mouse, views can be docked as fast
+
views in toolbars, grouped together in stacks, and dragged around to different areas.
+
Any view can also be maximized by double-clicking its title bar or pressing
+
Ctrl+M, or it can be minimized to show only its title bar.
+
  
 +
You will need to be root for most of these tasks.
  
 +
== Set up web content ==
  
 +
* Copy content from an existing server setup. Most of the content exists in CVS as well (<tt style="color:DarkGreen">[http://dev.eclipse.org/viewcvs/indextech.cgi/*checkout*/org.eclipse.emft/releng/common/ /cvsroot/technology/org.eclipse.emft/releng/common]</tt>):
  
With editors, you can likewise resize or organize in a number of ways. All editors
+
<table>
must be grouped together in one area, but they can be split horizontally or vertically
+
<tr><td><b>FROM</b> (emf.torolab.ibm.com)</td><td></td><td><b>TO</b> (emft.eclipse.org)</td></tr>
within that area to show multiple editors simultaneously. Editors can also be
+
<tr><td colspan="3"><hr noshade="noshade" size="1"/></td></tr>
maximized, by double-clicking the title bar or pressing Ctrl+M,
+
<tr><td>/home/www-data/build/emft</td><td>--&gt;</td><td>/home/www-data/build/emft</td></tr>
or minimized to show only its title bar.
+
<tr><td>/var/www/emft</td><td>--&gt;</td><td>/var/www/html/emft</td></tr>
 +
<tr><td>/var/www/technology/emft</td><td>--&gt;</td><td>/var/www/html/technology/emft</td></tr>
 +
</table>
  
 +
== Fix permissions & ownership ==
  
 +
<pre>
 +
cd /var/www/html; find . -type f -exec chmod 664 {} \;
 +
cd /var/www/html; find . -type d -exec chmod 775 {} \;
 +
cd /var/www/html; find . -exec chown apache:www {} \;
  
Many of these actions can be mapped to a key binding from the '''Workbench &gt;
+
cd /home/www-data/build; find . -type f -exec chmod 664 {} \;
Keys''' preference page. Look at the commands in the '''Window''' category
+
cd /home/www-data/build; find . -type d -exec chmod 775 {} \;
to see what commands can be bound to your own key sequences. Most
+
cd /home/www-data/build; find . -exec chown apache:www {} \;
part-manipulation commands are also found in the part context menu, accessible
+
cd /home/www-data/build/scripts; find . -type f -name "*.sh" -exec chmod 755 {} \;
by pressing Alt+- (Alt+minus).
+
</pre>
  
<hr><font size=-2>This FAQ was originally published in [http://www.eclipsefaq.org Official Eclipse 3.0 FAQs]. Copyright 2004, Pearson Education, Inc. All rights reserved. This text is made available here under the terms of the [http://www.eclipse.org/legal/epl-v10.html Eclipse Public License v1.0].</font>
+
<p>Apache 2 w/ PHP 5 was already installed. Only a few additional programs needed to be installed as well.</p>
 +
 
 +
== Install via <tt style="color:DarkGreen">yum</tt> using [http://people.redhat.com/~katzj/yum-xen.conf yum-xen.conf file] ==
 +
 
 +
<pre>yum -c http://people.redhat.com/~katzj/yum-xen.conf groupinstall "Java Development"
 +
yum -c http://people.redhat.com/~katzj/yum-xen.conf groupinstall "Development Tools"</pre>
 +
 
 +
== Install & symlink ==
 +
 
 +
* [http://w3.hursley.ibm.com/java/jim/ibmsdks/latest/ IBM JDK 1.4.2sr4-1] (symlink to <tt style="color:DarkGreen">/opt/ibm-java2-1.4</tt>)
 +
* [http://java.sun.com/j2se/1.5.0/download.jsp Sun JDK 1.5.0_06] (symlink to <tt style="color:DarkGreen">/opt/sun-java2-5.0</tt>)
 +
* [http://ant.apache.org/bindownload.cgi Apache Ant 1.6.5] (symlink to <tt style="color:DarkGreen">/opt/apache-ant-1.6</tt>)
 +
 
 +
== Fix web user (apache) ==
 +
 
 +
* Edit <tt style="color:DarkGreen">/etc/group</tt>. Add <tt style="color:DarkGreen">www</tt> group:
 +
 
 +
==== /etc/group ====
 +
<pre>www:x:501:user,apache</pre>
 +
 
 +
* Edit <tt style="color:DarkGreen">/etc/passwd</tt>. Change user's home directory and shell:
 +
 
 +
==== /etc/passwd ====
 +
<pre>apache:!!:48:48:Apache:/home/www-data:/bin/bash</pre>
 +
 
 +
* Edit <tt style="color:DarkGreen">/etc/sudoers</tt> to let you run commands and switch to the web user w/o needing a password. Add the following lines:
 +
 
 +
==== /etc/sudoers ====
 +
<pre>%www  ALL = (apache) NOPASSWD: ALL, (root) /usr/bin/su apache</pre>
 +
 
 +
* Switch to the web user. You should NOT be prompted for a password.
 +
 
 +
<pre>sudo -u apache bash</pre>
 +
 
 +
* Create an ssh key, WITH NO PASSPHRASE. Store in <tt style="color:DarkGreen">~/.ssh/id_rsa</tt> and <tt style="color:DarkGreen">~/.ssh/id_rsa.pub</tt>:
 +
<pre>ssh-keygen -b 2048 -t rsa</pre>
 +
 
 +
* Copy contents of <tt style="color:DarkGreen">~/.ssh/id_rsa.pub</tt> into ~/.ssh/authorized_keys file for user on dev.eclipse.org who
 +
will be running builds. This is so that the web user can commit changes to cvs
 +
(tagging, updating map files) for I, M, S & R builds.
 +
 
 +
* Test by ssh'ing to dev.eclipse.org, where <tt style="color:DarkGreen">_username_</tt> should be replaced with your actual username:
 +
 
 +
<pre>ssh _username_@dev.eclipse.org</pre>
 +
 
 +
* Run <tt style="color:DarkGreen">newgrp www</tt> so that when new files are created, they will use the group id <tt style="color:DarkGreen">www</tt> instead of <tt style="color:DarkGreen">apache</tt>:
 +
 
 +
<pre>newgrp www</pre>
 +
 
 +
* Set <tt style="color:DarkGreen">umask 022</tt> so that files will be created with group write perms 664 (see [[#/etc/bashrc]]):
 +
 
 +
<pre>umask 022</pre>
 +
 
 +
* Set the remote shell connection method for CVS to be <tt style="color:DarkGreen">ssh</tt> instead of the default <tt style="color:DarkGreen">rsh</tt> (see [[#/etc/bashrc]]):
 +
 
 +
<pre>export CVS_RSH=/usr/bin/ssh</pre>
 +
 
 +
* Set an <tt style="color:DarkGreen">ANT_HOME</tt> and <tt style="color:DarkGreen">JAVA_HOME</tt>, and add ant to the <tt style="color:DarkGreen">PATH</tt> (see [[#/etc/bashrc]]):
 +
 
 +
<pre>export ANT_HOME=/opt/apache-ant-1.6
 +
export JAVA_HOME=/opt/sun-java2-5.0
 +
export PATH=${PATH}:${ANT_HOME}/bin
 +
</pre>
 +
 
 +
* Switch to the root user.
 +
 
 +
* Append the following into <tt style="color:DarkGreen">/etc/bashrc</tt>, where <tt style="color:DarkGreen">_username_</tt> should be replaced with your actual dev.eclipse.org username:
 +
 
 +
==== /etc/bashrc ====
 +
<pre>umask 022
 +
export ANT_HOME=/opt/apache-ant-1.6
 +
export JAVA_HOME=/opt/sun-java2-5.0
 +
export PATH=${PATH}:${ANT_HOME}/bin
 +
export CVSROOT=:ext:_username_@dev.eclipse.org:/cvsroot/technology
 +
export CVS_RSH=/usr/bin/ssh</pre>
 +
 
 +
* Add the following to <tt style="color:DarkGreen">.bashrc</tt> and <tt style="color:DarkGreen">.bash_profile</tt> files:
 +
 
 +
==== ~/.bashrc ====
 +
<pre>if [ -f /etc/bashrc ]; then
 +
  . /etc/bashrc
 +
fi
 +
 
 +
if [ "$PS1" ]; then
 +
  # enable color support of ls and also add handy aliases
 +
  eval `dircolors -b`
 +
  alias ls='ls --color=auto'
 +
  alias ll='ls -l --color=auto'
 +
 
 +
  # set a fancy prompt
 +
  # 1;30 - grey, 1;31 - red, 1;32 - green, 1;33 - yellow, 1;34 - blue, etc.
 +
  PS1="\[\033[<b>1;30</b>m\]\u@\h:\w\\[\033[0;39m\] \$ "
 +
  export PS1=$PS1"\[\e]30;\u@\H:\w\a\]"
 +
  PATH=$PATH:/opt/ibm-java2-1.4/bin:/opt/apache-ant-1.6/bin
 +
  export PATH
 +
fi
 +
 
 +
source ~/.alias
 +
cat ~/.alias</pre>
 +
 
 +
==== ~/.bash_profile ====
 +
<pre>if [ -f ~/.bashrc ]; then
 +
  . ~/.bashrc
 +
fi
 +
 
 +
PATH=$PATH:$HOME/bin
 +
export PATH
 +
unset USERNAME</pre>
 +
 
 +
== Secure build script ==
 +
 
 +
* Since the webserver is public but builds should only be run by authorized users, we must secure access to the <tt style="color:DarkGreen">build.php</tt> script.
 +
 
 +
* Edit Apache config file to allow <tt style="color:DarkGreen">.htaccess</tt> rule changes to take affect. Change <tt style="color:DarkGreen">None</tt> to <tt style="color:DarkGreen">All</tt>:
 +
 
 +
==== /etc/httpd/conf/httpd.conf ====
 +
<pre># AllowOverride controls what directives may be placed in .htaccess files.
 +
# It can be "All", "None", or any combination of the keywords:
 +
#  Options FileInfo AuthConfig Limit
 +
#
 +
  AllowOverride All</pre>
 +
 
 +
* Create password file, where <tt style="color:DarkGreen">_username_</tt> should be replaced with an actual username and <tt style="color:DarkGreen">/path/to/password/file</tt> should be replaced with an actual path:
 +
 
 +
<pre>httpasswd -c /path/to/password/file _username_</pre>
 +
 
 +
* You can add additional users to the password file like this:
 +
 
 +
<pre>httpasswd /path/to/password/file _username2_</pre>
 +
 
 +
* Create <tt style="color:DarkGreen">.htaccess</tt> file, replacing <tt style="color:DarkGreen">/path/to/password/file</tt> with the same path used in the previous step:
 +
 
 +
==== /var/www/html/emft/.htaccess ====
 +
<pre>AuthType Basic
 +
AuthName "EMFT Build Server @ Eclipse.org"
 +
AuthUserFile /path/to/password/file
 +
Require valid-user</pre>
 +
 
 +
* Restart apache, eg.:
 +
 
 +
<pre>/usr/sbin/httpd -k restart
 +
- or -
 +
apache2ctl restart</pre>
 +
 
 +
* For more on htaccess, see [http://httpd.apache.org/docs/2.0/howto/auth.html http://httpd.apache.org/docs/2.0/howto/auth.html].
 +
 
 +
== Run a build ==
 +
 
 +
Using the username and password set up in the previous step, go here: [http://emft.eclipse.org/emft/build.php http://emft.eclipse.org/emft/build.php].
 +
 
 +
Check the build log while it's running or after it completes. If you see any messages such as <tt style="color:DarkGreen">permission denied</tt> or the following,
 +
something is amiss.
 +
 
 +
=== Debugging tips ===
 +
 
 +
<pre>execvp: Permission denied</pre>
 +
* Shell scripts must contain unix line endings. Run <tt style="color:DarkGreen">dos2unix</tt> to make sure, if copying them from a non-unix filesystem.
 +
* Shell scripts <tt style="color:DarkGreen">/home/www-data/build/emft/scripts/*.sh</tt> must be executable. See [[#Fix permissions & ownership]].
 +
 
 +
--
 +
 
 +
* Directories, eg. <tt style="color:DarkGreen">/home/www-data/build/emft/jet/downloads/drops/1.0.0</tt> must be writable by the web user
 +
(or group <tt style="color:DarkGreen">www</tt>). See [[#Fix permissions & ownership]].
 +
 
 +
--
 +
 
 +
<pre>eclipse.org: Connection refused
 +
cvs [checkout aborted]: end of file from server (consult above messages if any)
 +
</pre>
 +
* CVS connection refusals are the result of not being able to automatically ssh as the web user to _username_@dev.eclipse.org. This
 +
could be a problem with ssh keys (see above - [[#Fix web user (apache)]]) or the method CVS uses to
 +
connect (rsh instead of ssh - see [[#/etc/bashrc]]).
 +
 
 +
--
 +
 
 +
* For any compilation-related problems, missing file problems, or other issues not touched upon in this document, see [[EMFT_Procedures]].
 +
 
 +
== Display build logs, details & test results ==
 +
 
 +
Builds - including unpublished Nightly builds - are listed on the downloads page here: [http://emft.eclipse.org/technology/emft/downloads/ http://emft.eclipse.org/technology/emft/downloads/].
 +
 
 +
Published builds are located here: [http://downloads.eclipse.org/technology/emft/downloads/ http://downloads.eclipse.org/technology/emft/downloads/].
 +
 
 +
== Add additional users ==
 +
 
 +
* As root, run the following script:
 +
 
 +
<pre>#!/bin/bash
 +
 
 +
if [ $# -lt 1 ]; then
 +
  echo "Usage: ./adduser <username>"
 +
else
 +
  last=`tail -2 /etc/group | head -1`; last=${last##*:x:}; last=${last%%:*}; (( last++ )); #echo $last
 +
  newgrp=$1":x:"$last":"$1;
 +
 
 +
  echo -n "Adding: "
 +
  echo $1"::"$last":"$last"::/home/"$1":/bin/bash"
 +
  echo $1"::"$last":"$last"::/home/"$1":/bin/bash" >> /etc/passwd
 +
 
 +
  wwwgrp=`tail -1 /etc/group`","$1; #echo $wwwgrp;
 +
  wc=`wc -l /etc/group`; wc=${wc%% *}; (( wc-- )); wc="-"$wc; #echo $wc;
 +
  head $wc /etc/group > /etc/group.new
 +
 
 +
  echo -n "Adding: "
 +
  echo $newgrp
 +
  echo $newgrp >> /etc/group.new
 +
 
 +
  echo -n "Adding: "
 +
  echo $wwwgrp
 +
  echo $wwwgrp >> /etc/group.new
 +
 
 +
  mv /etc/group.new /etc/group
 +
 
 +
  echo $1 | passwd --stdin $1
 +
  echo "Creating home dir: /home/$1"
 +
  cp -r /home/user /home/$1
 +
  echo "Setting ownership: $1:$1"
 +
  chown -R $1:$1 /home/$1
 +
fi</pre>
 +
 
 +
--[[User:Nickb|Nick Boldt]] 21:48, 14 March 2006 (EST)

Revision as of 22:48, 14 March 2006

EMFT Build Server Setup

You will need to be root for most of these tasks.

Set up web content

FROM (emf.torolab.ibm.com)TO (emft.eclipse.org)

/home/www-data/build/emft-->/home/www-data/build/emft
/var/www/emft-->/var/www/html/emft
/var/www/technology/emft-->/var/www/html/technology/emft

Fix permissions & ownership

cd /var/www/html; find . -type f -exec chmod 664 {} \;
cd /var/www/html; find . -type d -exec chmod 775 {} \;
cd /var/www/html; find . -exec chown apache:www {} \;

cd /home/www-data/build; find . -type f -exec chmod 664 {} \;
cd /home/www-data/build; find . -type d -exec chmod 775 {} \;
cd /home/www-data/build; find . -exec chown apache:www {} \;
cd /home/www-data/build/scripts; find . -type f -name "*.sh" -exec chmod 755 {} \;

Apache 2 w/ PHP 5 was already installed. Only a few additional programs needed to be installed as well.

Install via yum using yum-xen.conf file

yum -c http://people.redhat.com/~katzj/yum-xen.conf groupinstall "Java Development"
yum -c http://people.redhat.com/~katzj/yum-xen.conf groupinstall "Development Tools"

Install & symlink

Fix web user (apache)

  • Edit /etc/group. Add www group:

/etc/group

www:x:501:user,apache
  • Edit /etc/passwd. Change user's home directory and shell:

/etc/passwd

apache:!!:48:48:Apache:/home/www-data:/bin/bash
  • Edit /etc/sudoers to let you run commands and switch to the web user w/o needing a password. Add the following lines:

/etc/sudoers

%www  ALL = (apache) NOPASSWD: ALL, (root) /usr/bin/su apache
  • Switch to the web user. You should NOT be prompted for a password.
sudo -u apache bash
  • Create an ssh key, WITH NO PASSPHRASE. Store in ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub:
ssh-keygen -b 2048 -t rsa
  • Copy contents of ~/.ssh/id_rsa.pub into ~/.ssh/authorized_keys file for user on dev.eclipse.org who

will be running builds. This is so that the web user can commit changes to cvs (tagging, updating map files) for I, M, S & R builds.

  • Test by ssh'ing to dev.eclipse.org, where _username_ should be replaced with your actual username:
ssh _username_@dev.eclipse.org
  • Run newgrp www so that when new files are created, they will use the group id www instead of apache:
newgrp www
  • Set umask 022 so that files will be created with group write perms 664 (see #/etc/bashrc):
umask 022
  • Set the remote shell connection method for CVS to be ssh instead of the default rsh (see #/etc/bashrc):
export CVS_RSH=/usr/bin/ssh
  • Set an ANT_HOME and JAVA_HOME, and add ant to the PATH (see #/etc/bashrc):
export ANT_HOME=/opt/apache-ant-1.6
export JAVA_HOME=/opt/sun-java2-5.0
export PATH=${PATH}:${ANT_HOME}/bin
  • Switch to the root user.
  • Append the following into /etc/bashrc, where _username_ should be replaced with your actual dev.eclipse.org username:

/etc/bashrc

umask 022
export ANT_HOME=/opt/apache-ant-1.6
export JAVA_HOME=/opt/sun-java2-5.0
export PATH=${PATH}:${ANT_HOME}/bin
export CVSROOT=:ext:_username_@dev.eclipse.org:/cvsroot/technology
export CVS_RSH=/usr/bin/ssh
  • Add the following to .bashrc and .bash_profile files:

~/.bashrc

if [ -f /etc/bashrc ]; then
  . /etc/bashrc
fi

if [ "$PS1" ]; then
  # enable color support of ls and also add handy aliases
  eval `dircolors -b`
  alias ls='ls --color=auto'
  alias ll='ls -l --color=auto'

  # set a fancy prompt
  # 1;30 - grey, 1;31 - red, 1;32 - green, 1;33 - yellow, 1;34 - blue, etc.
  PS1="\[\033[<b>1;30</b>m\]\u@\h:\w\\[\033[0;39m\] \$ "
  export PS1=$PS1"\[\e]30;\u@\H:\w\a\]"
  PATH=$PATH:/opt/ibm-java2-1.4/bin:/opt/apache-ant-1.6/bin
  export PATH
fi

source ~/.alias
cat ~/.alias

~/.bash_profile

if [ -f ~/.bashrc ]; then
  . ~/.bashrc
fi

PATH=$PATH:$HOME/bin
export PATH
unset USERNAME

Secure build script

  • Since the webserver is public but builds should only be run by authorized users, we must secure access to the build.php script.
  • Edit Apache config file to allow .htaccess rule changes to take affect. Change None to All:

/etc/httpd/conf/httpd.conf

# AllowOverride controls what directives may be placed in .htaccess files.
# It can be "All", "None", or any combination of the keywords:
#   Options FileInfo AuthConfig Limit
#
  AllowOverride All
  • Create password file, where _username_ should be replaced with an actual username and /path/to/password/file should be replaced with an actual path:
httpasswd -c /path/to/password/file _username_
  • You can add additional users to the password file like this:
httpasswd /path/to/password/file _username2_
  • Create .htaccess file, replacing /path/to/password/file with the same path used in the previous step:

/var/www/html/emft/.htaccess

AuthType Basic
AuthName "EMFT Build Server @ Eclipse.org"
AuthUserFile /path/to/password/file
Require valid-user
  • Restart apache, eg.:
/usr/sbin/httpd -k restart
 - or -
apache2ctl restart

Run a build

Using the username and password set up in the previous step, go here: http://emft.eclipse.org/emft/build.php.

Check the build log while it's running or after it completes. If you see any messages such as permission denied or the following, something is amiss.

Debugging tips

execvp: Permission denied
  • Shell scripts must contain unix line endings. Run dos2unix to make sure, if copying them from a non-unix filesystem.
  • Shell scripts /home/www-data/build/emft/scripts/*.sh must be executable. See #Fix permissions & ownership.

--

  • Directories, eg. /home/www-data/build/emft/jet/downloads/drops/1.0.0 must be writable by the web user

(or group www). See #Fix permissions & ownership.

--

eclipse.org: Connection refused
cvs [checkout aborted]: end of file from server (consult above messages if any)
  • CVS connection refusals are the result of not being able to automatically ssh as the web user to _username_@dev.eclipse.org. This

could be a problem with ssh keys (see above - #Fix web user (apache)) or the method CVS uses to connect (rsh instead of ssh - see #/etc/bashrc).

--

  • For any compilation-related problems, missing file problems, or other issues not touched upon in this document, see EMFT_Procedures.

Display build logs, details & test results

Builds - including unpublished Nightly builds - are listed on the downloads page here: http://emft.eclipse.org/technology/emft/downloads/.

Published builds are located here: http://downloads.eclipse.org/technology/emft/downloads/.

Add additional users

  • As root, run the following script:
#!/bin/bash

if [ $# -lt 1 ]; then
  echo "Usage: ./adduser <username>"
else
  last=`tail -2 /etc/group | head -1`; last=${last##*:x:}; last=${last%%:*}; (( last++ )); #echo $last
  newgrp=$1":x:"$last":"$1;

  echo -n "Adding: "
  echo $1"::"$last":"$last"::/home/"$1":/bin/bash"
  echo $1"::"$last":"$last"::/home/"$1":/bin/bash" >> /etc/passwd

  wwwgrp=`tail -1 /etc/group`","$1; #echo $wwwgrp;
  wc=`wc -l /etc/group`; wc=${wc%% *}; (( wc-- )); wc="-"$wc; #echo $wc;
  head $wc /etc/group > /etc/group.new

  echo -n "Adding: "
  echo $newgrp
  echo $newgrp >> /etc/group.new

  echo -n "Adding: "
  echo $wwwgrp
  echo $wwwgrp >> /etc/group.new

  mv /etc/group.new /etc/group

  echo $1 | passwd --stdin $1
  echo "Creating home dir: /home/$1"
  cp -r /home/user /home/$1
  echo "Setting ownership: $1:$1"
  chown -R $1:$1 /home/$1
fi

--Nick Boldt 21:48, 14 March 2006 (EST)

Back to the top