(C)Copyright 1998-2003 Marvell(R). All rights reserved =========================================================================== tcge.txt created 14-Feb-2003 Readme File for tcge v6.03 3Com (3C2000/3C940) Gigabit Adapter Family driver for Solaris This file contains 1 Overview 2 Installation 2.1 Installation using "pkgadd" 2.2 Installation using "admintool" 2.3 Procedures during installation 2.4 Deinstallation 2.5 Adding adapters 3 Driver parameters 3.1 Port parameters 3.2 Adapter parameters 3.3 Global driver parameters 4 VLAN configuration and parameters 4.1 VLAN configuration using the script "tcge_vlan_config" 4.2 Manual VLAN configuration 5 Tuning 6 "ndd" support 7 Virtual Cable Tester(TM) (VCT) 8 Troubleshooting 9 Error messages =========================================================================== 1 Overview =========== The tcge driver supports the 3Com (3C2000/3C940) Gigabit Adapter Family on Solaris 7 and higher for x86 platforms with a PCI 32 bit bus. *** 2 Installation =============== Before installing the 3C940sol_x86v.tar.Z driver package, the package has to be copied from the 3Com installation CD-ROM or downloaded from the 3Com web site to a directory of your choice on your system. Then proceed with the following steps: 1. Unpack the file 3C940sol_x86v.tar.Z with the command: uncompress 3C940sol_x86v.tar.Z The unpacked file will have the following format: 3C940sol_x86v.tar 2. Untar this file with the command: tar -xvf 3C940sol_x86v.tar The result is the subdirectory "sol3C940" containing the driver package. In your actual directory you will also find the driver readme file as a text version (tcge.txt) and as a HTML version (tcge.htm). There are two tools available for installation: 'pkgadd' and 'admintool'. 'pkgadd' runs from the command line, while 'admintool' uses the graphical interface. NOTE: If you want to configure an adapter for the use of VLANs, you have to do this after driver installation is finished. Please refer to section 4 for details. Now you are able to install the driver package on your system. 2.1 Installation using "pkgadd" -------------------------------- There are two possibilities to install the driver using 'pkgadd': - Manual installation (user input required) - Automatic installation (no user input required) Manual installation with "pkgadd" --------------------------------- To install the driver using 'pkgadd', proceed as follows: 1. Go to the directory where the driver subdirectory "sol3C940" is located. 2. Execute 'pkgadd': pkgadd -d . sol3C940 A shell window will come up and you will be asked whether you want to configure IP interfaces during installation or not. 3. If yes, enter name, IP address and network number for every interface you want to set up (in case you have more than one adapter installed on your system). After 'pkgadd' has run, the adapter is fully functional. 4. If no, only the driver will be loaded and you have to configure all interfaces manually. In both cases the system prompts you to reboot after successful installation, but this may be ignored. Automatic installation with "pkgadd" ------------------------------------ During automatic installation, no input from the user is required. After 'pkgadd' has run, the driver is added to the system, but no IP interfaces have been attached to it. You have to do this manually after installation is finished. To start automatic installation, proceed as follows: 1. Go to the directory where the driver subdirectory "sol3C940" is located. 2. To suppress user interaction, create a response file named response in the working directory (or choose any other name and/or location) that is used by 'pkgadd' during installation: Execute: touch response 3. The file must exist but remains empty. 4. Now check the admin file /var/sadm/install/admin/default for the following entry: action=ask 5. This entry has to be set to action=nocheck Otherwise you will be prompted to allow execution of commands that need root authority during installation. 6. The file /var/sadm/install/admin/default is not writable. To assign the value above, create a new admin file named default in the working directory (or choose any other name and/or location). For more information on the admin file, refer to the man page admin(4). 7. Execute the 'pkgadd' command with the following options: pkgadd -d . -r ./respone -a ./default sol3C940 8. If necessary exchange ./response and/or ./default with the name and/or location you have chosen for the response and/or admin file. Now the package will be installed without any further input needed. After successful installation the system will prompt you to reboot, but this may be ignored. 2.2 Installation using "admintool" ----------------------------------- To install the driver using 'admintool', proceed as follows: 1. Start 'admintool'. 2. Select Browse->Software. 3. Select Edit->Add. 4. In the dialog box, select the location of the "sol3C940" subdirectory. The left panel shows the available software packages. 5. Select "3Com (3C2000/3C940) Gigabit Adapter Family". 6. Press "Add". A shell window will come up and you will be asked whether you want to configure IP interfaces during installation or not: 7. If yes, enter name, IP address and network number for every interface you want to set up (in case you have more than one adapter installed on your system). Once you have entered all information, every adapter with a configured interface is fully functional. 8. If no, only the driver will be loaded and you have to configure all interfaces manually. In both cases 'admintool' will prompt you to reboot after successful installation, but this may be ignored. 2.3 Procedures during installation ----------------------------------- This is a short description of what happens during installation. This information is not needed to install and use the driver but it may be useful if any problems occur: - The driver software package is added to the Solaris package database. - The driver binary is copied to /kernel/drv. - The sample configuration file tcge.conf is copied to /kernel/drv. - The startup script S50_tcge for modifying network settings is copied to /etc/rcS.d. - The VLAN configuration script tcge_vlan_config is copied to /usr/sbin. - The man page tcge.7d is copied to /usr/share/man/man7d. - The driver is added to the system and loaded with the add_drv command. - If you do not choose automatic installation, the following entries are made for each adapter: -- a line with the IP address and interface name is added to /etc/hosts -- a line with the network address and netmask is added to /etc/netmasks -- a file /etc/hostname.tcgeX is created, where X is the number to which the adapter is attached. The file contains only one line with the name of the corresponding interface. - If you do not choose automatic installation, the IP interface(s) is (are) started with the 'ifconfig' command. 2.4 Deinstallation ------------------- NOTE: In case you have changed the configuration settings in the tcge.conf file and you want to keep the settings, make a backup file before deinstalling the driver. To remove the driver, proceed as follows: 1. Enter: pkgrm sol3C940 2. Remove all according lines with the IP addresses in /etc/hosts. 3. Remove all according lines with the netmasks in /etc/netmasks. 2.5 Adding adapters -------------------- There are two ways of adding additional adapters to an existing installation: - Remove the driver with 'pkgrm', insert the additional adapters, and then reinstall the driver with 'pkgadd'. - Add the necessary adapters manually. Using 'pkgrm' and 'pkgadd' -------------------------- To add additional adapters using 'pkgrm', proceed as follows: 1. Enter : pkgrm sol3C940 You do not need to clean up /etc/hosts, /etc/netmasks, and the configuration file. 2. Insert the additional adapter(s). 3. Reinstall the driver with 'pkgadd'. 4. Use the same interface name(s) as before. The corresponding entries will be found and can be reused. 5. In case the addresses have been swapped after reinstallation, swap the numbers of the corresponding /etc/hostname.tcgeX files (or simply swap the cables). 6. Reboot (may not be done). Manual addition --------------- To manually add additional adapters, proceed as follows: 1. Look in /dev for the devices tcge* before and after installation of the additional adapter(s) to find out which instance number to use for the hostname.tcgeX file. 2. Insert the additional adapter(s). 3. Enter the necessary entries for each additional adapter (IP address, interface name, network address, netmask) in the corresponding files as described above under "Procedures during installation". 4. Reboot (may not be done). 5. Look in /dev for the devices tcge* (see step 1). *** 3 Driver parameters ==================== Parameters can be set in a file called tcge.conf in the directory /kernel/drv. This file is created during installation, but does only contain comments. Edit it to review your settings. The syntax for this file is (see also: 'man driver.conf'): - for string parameters: ParamName="string"; - for integer parameters: ParamName=value; WARNING: All parameters and values are case sensitive. Write them exactly as shown here! All parameters have to be followed by a semicolon! Parameters in this file will be used immediately if you create this file before installing the driver (Installation will then ask to overwrite the file). If you change it while the driver is already running, you have three possibilities to use the new settings: - Reboot the system. - Unload the driver with rem_drv and load it again with add_drv. Please refer to section 4.1 for details about these two ways. - Unload and reload the driver with modunload/modload. Unload and reload the driver with modunload/modload --------------------------------------------------- To use this possibility, proceed as follows: 1. Deactivate the IP interfaces of all adapters with the following command executed for each interface: ifconfig unplumb Example: ifconfig tcge0 unplumb 2. Make sure that no other application or OS daemon is connected to any of the 3C940 adatpers. 3. First you need the module ID of the 3C940 driver. You can retrieve it by entering the following command: modinfo | grep tcge You will see one line like this: "69 1026bb64 322bb 67 1 tcge ... The module ID is the first value in the row above, in our example "69". This ID is needed to unload the driver with the command modunload. 4. Remove the driver module from the kernel with the command: modunload -i If there are any remaining connections to any of the adapters (see steps 1 and 2), you will see the following error messages: "can't unload the module: Device busy" 5. Exchange with the value from step 3. 6. Load the driver again into the kernel with the following command: modload tcge For more details about modinfo, modunload or modload, please refer to the manual pages by executing the commands 'man modinfo', 'man modunload' or 'man modload' accordingly. 7. After loading the driver, set up the IP interfaces manually using the ifconfig command. To start the interfaces, enter the following command for each interface: ifconfig tcge plumb up 8. Exchange with the interface number. 9. Exchange with the hostname you have chosen in /etc/hosts for the IP address. 3.1 Port parameters -------------------- These settings belong to the port of the adapter. In the following description, 'X' stands for the instance number of the adapter. AutoNegotiation_A_InstX ----------------------- Type: string Default value: "On" Valid values: "On" "Off" DuplexCapabilities_A_InstX -------------------------- Type: string Default value: "Both" Valid values: "Half" "Full" "Both" "Both": Port can connect with full-duplex and half-duplex. "Full": Port connects with full-duplex. "Half": Port connects with half-duplex. If auto-negotiation is set to "On", all three values are possible. If it is set to "Off", only "Full" and "Half" are allowed. This parameter is useful if your link partner does not support all possible combinations. FlowControl_A_InstX ------------------- Type: string Default value: "SymOrRem" Valid values: "Sym" "SymOrRem" "LocSend" "None" This parameter can be used to set the flow control capabilities the port reports during auto-negotiation. The values represent the following configurations: - "SymOrRem" = SymmetricOrRemote Both or only the remote link partner are allowed to send PAUSE frames (possible results: symmetrical flow control, asymmetrical flow control towards local station, no flow control). - "Sym" = Symmetric Both link partners are allowed to send PAUSE frames (possible results: symmetrical flow control, no flow control). - "LocSend" = LocalSend Asymmetrical flow control to other station: only the local link partner is allowed to send PAUSE frames (possible results: flow control to other station, no flow control). - "None" No link partner is allowed to send PAUSE frames (possible result: no flow control). NOTE: This parameter is ignored if auto-negotiation is set to "Off". Role_A_InstX ------------ Type: string Default value: "Auto" Valid values: "Auto" "Master" "Slave" NOTE: If auto-negotiation is set to "Off", the correct role must be set manually. This parameter defines the role of the port for the physical clock generation. In order for two 1000Base-T ports to communicate, one must take the role as master (providing timing information), while the other must be slave. Usually, this is negotiated between the two ports during link establishment. If this fails, a port can be forced to a specific setting with this parameter. Speed_A_InstX ------------- Type: string Default value: "Auto" Valid values: "Auto" "1000" "100" "10" This parameter sets the link speed of the port to the specified value. If you choose "Auto", the adapter will negotiate the speed with the link partner automatically. 3.2 Adapter parameters ----------------------- In the following description, 'X' stands for the instance number of the according adapter. JumboFrames_InstX ----------------- Type: string Default value: "Off" Valid values: "On" "Off" To enable support for Jumbo Frames (frames with a length of up to 9014 bytes or up to 9018 bytes in case of VLAN frames), set this parameter to "On". Using Jumbo Frames can speed up network throughput, because longer frames reduce the overhead in the operating system. For full Jumbo Frame support, the MTU (Maximum Transfer Unit) size used by TCP/IP must also be changed. This can be done with the 'ifconfig' command. To change the MTU size, proceed as follows: 1. The file /etc/rcS.d/S50_tcge contains a line to set MTU size during system start. Edit this file. 2. Remove the "#" before the ifconfig line. 3. If necessary, change the adapter device number from tcge0 to the attached number displayed during driver startup. 4. Set the MTU size to 9000, the 14 bytes of the MAC address header or 18 bytes in case of a VLAN MAC address header are not counted. NOTE: Jumbo Frames can only be used if all the equipment in your subnetwork support Jumbo Frames (many current switches do not)! Devices without Jumbo Frame Support will simply drop the longer frames (and possibly report them as error frames). If problems occur, try to connect two 3C940 adapters (with Jumbo Frames enabled) back-to-back. RxRingSize_InstX ---------------- Type: integer Default value: 128 Valid range: 100 - 5000 TxRingSize_InstX ---------------- Type: integer Default value: 128 Valid range: 50 - 2500 WARNING: If you want to increase the size of the transmit and/or receive descriptor rings, bear in mind that every descriptor needs about 2 KByte of memory for normal ethernet frames and about 10 KByte of memory for jumbo frames. If you choose the maximum ring sizes for transmit and receive descriptor rings and you have only one adapter installed, you would need about 150 MByte of memory! So it is easily possible that you need too much system memory for your installed adapter(s)! Please check the amount of memory you would need for your chosen ring sizes on all installed adapters and compare this value with the memory size of your machine (you can obtain this value with the command 'prtconf | more'). CopyThreshold_InstX ------------------- Type: integer Default value: 1500 Valid range: 0 - 1500 During transmit, the driver needs the physical memory address of frames to inform the hardware where to find the frame data. Setting the DMA address on Solaris is rather slow, so in many cases it is faster to copy the frame data to a buffer that has been set up in advance during driver load. All frames with a length <= CopyThreshold are copied to such buffers. For longer frames, the real DMA setup is executed. By default (without Jumbo Frame support!), all frames are copied. You can experiment with this parameter to find out if your sytem performs better with only smaller frames copied. Ignore_LenErr_InstX ------------------- Type: string Default value: "Off" Valid values: "Off" "On" Ignore_LenErr_InstX can be used as a workaround for the EDP frame problem. EDP (Extreme Discovery Protocol) is a proprietary protocol that is used by Extreme Gigabit switches to exchange information about connected switches etc. These frames (at least the ones we have seen on a Extreme Summit 1) have a length field indicating 316 bytes, but a real frame length of 338 bytes. Such frames are counted by the MAC in the InRangeLength error counter, which is one of the counters that is summed up to form the counter "input errors". The counter even goes up if the frames are received only in the MAC and dropped immediately. These frames go to a constant MAC address of 00:e0:2b:00:00:00, which is normally not received by the driver, except in promiscuous mode. In the output of 'netstat -I tcgeX', these errors are visible as "input errors", going up two times per second in bigger installations (multiple Extreme switches). To avoid this, the parameter Ignore_LenErr_InstX has been introduced. If it is set to "On" in the configuration file tcge.conf, the InRangeLength errors are not added to the "input errors". Note that the frames are in fact incorrect and so they must be counted. But if you are puzzled by the error counter going up, you can set this parameter. To verify if you are really have this kind of problem, the InRangeLength error counter can be viewed by using 'netstat -k tcgeX'. The error will show up under the label "inrangeerr". DescrPollTime_InstX ------------------- Type: integer Default value: 250 Valid values: 0 to 250 DescrPollTime_InstX can be used as a workaround for a possible hang-up of the data transmission over one link due to a hardware timing problem. Please do not change the default value without contacting the 3Com support for further advice! 3.3 Global driver parameters ----------------------------- These parameters will apply to all supported 3C940 adapters installed on the system. DisplayMessages --------------- Type: string Default value: "Yes" Valid values: "Yes" "No" Enable or disable extensive messages during driver load and link up. *** 4 VLAN configuration and parameters ==================================== NOTE: If you want to configure VLANs for an adapter, all traffic on the adapter has to be handled by VLANs! Mixing VLAN interfaces with non VLAN interfaces connected to the same adapter is not allowed! A different adapter can be configured as a non VLAN adapter at the same time. The complete configuration is executed in the file /kernel/drv/tcge.conf. A sample file can be found at this location after the normal driver installation process without VLAN parameters. There are two possibilites to configure VLANs for an adapter: - Use the script tcge_vlan_config. - Edit the configuration file /kernel/drv/tcge.conf manually. NOTE: We recommend to use the script! It is much easier than the manual configuration. 4.1 VLAN configuration using the script "tcge_vlan_config" ----------------------------------------------------------- The script tcge_vlan_config was copied into the directory /usr/sbin during driver installation. NOTE: You have to be logged in as user 'root' to execute it. With this script you are able to configure any 3Com Gigabit Ethernet Adapter previously installed on the system for VLAN support. To configure VLANs using tcge_vlan_config, proceed as follows: 1. Before you start the script, you have to find out the instance number(s) of the adapter(s) you want to configure as VLANS. List all network interfaces with: ifconfig -a 2. Search for interfaces named tcge0, tcge1 and so on. The according IP address will show you the correct adapter. The value of in the interface name tcge reflects the instance number of the adapter: Instance number belonging to tcge0: 0 Instance number belonging to tcge1: 1 ... After you have determined the adapters you want to configure, configuration with tcge_vlan_config can be started: 3. Start the script by entering: tcge_vlan_config at the command line. The script will prompt you to define the following parameters: 4. Instance number: Enter the instance number of the adapter to configure. 5. VLAN ID: Enter a VLAN ID for each VLAN. Allowed values range from 1 to 4094. The IDs distinguish the VLANs in your network topology. NOTE: Please be careful: switches that are VLAN capable often have a default VLAN configured. Do not choose the same VLAN ID as used in the switch or change the ID of the switch default VLAN. 6. Jumbo Frame Support: Decide whether the VLAN is to support Jumbo frames or not. If not all VLANs for the same adapter need jumbo frame support, this is no problem. The adapter will be configured accordingly. NOTE: It is not necessary to set the MTU size for jumbo frames in the file /etc/rcS.d/S50_tcge manually! The script will do this for you. 7. Hostname: Enter the hostname to be used for the VLAN IP interface. 8. IP Address: Enter the IP address for the VLAN IP interface. 9. Netmask: Enter the netmask for the VLAN IP interface. When you have finished defining the parameters, the script adds all necessary entries to the file /kernel/drv/tcge.conf, adds all entries to the files /etc/hosts and /etc/netmasks and creates all /etc/hostname. files. When the configuration is finished, you have two possibilities to activate the changes in tcge.conf: - Reboot the system. - Unload the driver with rem_drv and load it again with add_drv. NOTE: If you execute all changes in tcge.conf after package installation and before you reboot the system, no additional steps as described below have to be performed. NOTE: Before you choose one of the above possibilities, you MUST remove the device links to the driver in the /dev directory manually! Otherwise these links and the entries in the /devices directory will not be created properly to show the new configuration of the driver. Enter the following command to remove the device links: rm /dev/tcge*[0-9] WARNING: Please be very careful! It is possible to remove other devices and to damage the system! Now all entries are removed. Reboot the system ----------------- If you want to reboot the system, proceed as follows: 1. Remove the links to the driver in the /dev directory as described above. 2. Create the file "reconfigure" in the root directory with the following command: touch /reconfigure This informs the system about the nessecitiy to recreate the /dev and /devices directories. Unload and reload the driver with rem_drv/add_drv ------------------------------------------------- To use the second possibility, proceed as follows: 1. Deactivate the IP interfaces of all 3C940 adapters with the following command executed for each interface: ifconfig unplumb Example: ifconfig tcge0 unplumb 2. Make sure that no other application or OS daemon is connected to any of the 3C940 adatpers. 3. Remove the driver from the system with the command: rem_drv tcge If there are any remaining connections to any of the 3C940 adapters (see step 2), you will see the following error messages: "Device busy Cannot unload module: tcge Will be unloaded upon reboot." 4. Load the driver again into the system with the following command: add_drv -f -c pci -m '* 0660 root sys' -i '"pci10b7,0010" "pci10b7,1700"' tcge WARNING: Please make sure that the whole add_drv command is on one line! The easiest way to achive this is to write a script. For more details about rem_drv or add_drv, please refer to the manual pages by executing the commands 'man rem_drv' or 'man add_drv' accordingly. 5. After loading the driver, set up the IP interfaces manually using the 'ifconfig' command. or 6. Reboot the system and the IP interfaces will be set automatically. VLAN IP interface naming ------------------------ Example: You have configured two VLANs for instance 0. The values you have chosen might have been the following: First VLAN: ID: 2 Jumbo frame support: Off Second VLAN: ID: 4 Jumbo frame support: On The entry in /kernel/drv/tcge.conf created with the script will look like this: # BEGIN VLAN configuration settings for instance 0 Vlan_Inst0_Enable="Yes"; Vlan_0_DevNum=0; Vlan_0_ID=2; Vlan_0_Jumbo="Off"; Vlan_100_DevNum=100; Vlan_100_ID=4; Vlan_100_Jumbo="On"; Vlan_Inst0_Count=2; # END VLAN configuration settings for instance 0 NOTE: If you want to disable VLAN support at a later time, only switch Vlan_Inst_Enable to "No". It is not necessary to delete all the settings. For each VLAN you have to set up one interface. The number of the interface must be the same as the value of the according Vlan_X_DevNum parameter. For our example you need the following two values: Vlan_0_DevNum=0 This value belongs to the VLAN with ID 2. Vlan_100_DevNum=100 This value belongs to the VLAN with ID 4. The interface for the first VLAN will be tcge0, for the second tcge100. To start the VLAN interfaces, proceed as follows: 1. Enter the commands: ifconfig tcge0 plumb broadcast + netmask + up ifconfig tcge100 plumb mtu 9000 broadcast + netmask + up 2. Exchange with the hostname you have chosen for the VLAN. 3. Now enter 'ifconfig -a' again to check if all interfaces are set up correctly. 4. Now you can try to ping to another machine to check whether or not the connection works. Additionally, you have the possibility to check the VLAN settings for a specific interface by use of the 'ndd' command. If you want to know the settings for tcge100 from the example above, enter the command: ndd /dev/tcge100 vlan_props If VLAN support is enabled on the according interface, you get a list of three parameters (used in tcge.conf): - VLAN number: This is the number of the VLAN, for tcge100 it is 100. - VLAN ID: This is the VLAN ID you have chosen. - VLAN Jumbo Frames: Did you choose jumbo frame support? If VLAN support is disabled, you get the following message: "No VLAN support on tcgeX enabled." X is the number of the according interface. For our example the output will look like the following: VLAN number: 100 VLAN ID: 4 VLAN Jumbo Frames: Yes For more details about 'ndd' refer to section 6. 4.2 Manual VLAN configuration ------------------------------ The complete VLAN configuration will be carried out in the driver configuration file /kernel/drv/tcge.conf. Please refer to 'man driver.conf' for further information about these files. An excerpt from tcge.conf looks like the following: # # Configuration file for the tcge VLAN driver. # See tcge.txt or tcge.htm for a description of the parameters. # Uncomment and change the settings you need. # # The decimal value in _Inst0 in all parameters is the instance # number of the appropriate adapter. Set it to needed value. # ..._Inst0 means instance number 0, ..._Inst1 means instance number # 1 and so on. # # WARNING: All parameters and values are case sensitive. Write them # exactly as shown here! # All parameters have to be followed by a semicolon! # AutoNegotiation: Values are: On, Off; Default = On # AutoNegotiation_A_Inst0="On"; # DuplexCapabilities: Values are: Half, Full, Both; Default = Both # DuplexCapabilities_A_Inst0="Both"; If you want to change the default values, you only have to uncomment the appropriate parameter and set it to the desired value. Here is an example how the VLAN parameters will look like: Vlan_Inst0_Enable="Yes"; Vlan_0_DevNum=0; Vlan_0_ID=2; Vlan_0_Jumbo="Off"; Vlan_100_DevNum=100; Vlan_100_ID=4; Vlan_100_Jumbo="On"; Vlan_Inst0_Count=2; NOTE: All parameters and values are case sensitive. Write them exactly as shown here! All parameters have to be followed by a semicolon! If you want to configure an adapter for the use of VLANs, proceed as follows: 1. The first parameter you have to set is: Vlan_Inst_Enable="Yes"; is a placeholder that has to be replaced with the actual instance number of the adapter given by the system. Without this parameter no further VLAN parameters will be read by the driver for this adapter. NOTE: If you want to disable VLAN support at a later time, only switch Vlan_Inst_Enable to "No". It is not necessary to delete or uncomment all the settings. 2. For each VLAN you want to create, you MUST set two parameters: Vlan__DevNum; Vlan__ID; is a placeholder that has to be exchanged with the real VLAN number. These numbers have to be set in a special manner. For each adapter there is a numbering scheme that MUST be used: The first VLAN you create has the same number as the instance number given to the adapter by the system. For every additional VLAN simply add 100 to the instance number. Example: Instance number of the adapter: 0 First VLAN to create: Vlan_0_DevNum, Vlan_0_ID Second VLAN: Vlan_100_DevNum, Vlan_100_ID Third VLAN: Vlan_200_Dev_num, Vlan_200_ID ... and so on. Instance number of the adapter: 1 First VLAN to create: Vlan_1_DevNum, Vlan_1_ID Second VLAN: Vlan_101_DevNum, Vlan_101_ID Third VLAN: Vlan_201_Dev_num, Vlan_201_ID ... and so on. NOTE: This is necessary to create device minor nodes in the driver that have the same numbers as the according interfaces you have to create with 'ifconfig'. More details, see below. PARAMETERS ---------- Parameter: Vlan__DevNum ------------------------------- Type: integer Default value: None! You MUST set this value! Valid Values: Not limited Example: Vlan_0_DevNum="0"; Vlan_100_DevNum="100"; This parameter defines the number tcge of the interface that belongs to the VLAN with the parameter Vlan__DevNum. The value for Vlan__DevNum should be . You are free to choose a different value. We recommend to use because it is very easy to distinguish all interfaces belonging to the same board since the last digit is the same for all interfaces. Parameter: Vlan__ID --------------------------- Type: integer Default value: None! You MUST set this value! Valid values: 1 to 4094 Example: Vlan_0_ID=2; Vlan_100_ID=122; This parameter defines the VLAN ID as specified in IEEE 802.1Q. It is used to distinguish the VLANs in VLAN capable drivers and switches. For this reason it is not allowed to use a chosen VLAN ID on the same adapter again! The value for Vlan__ID can be any value in the range from 1 to 4094. NOTE: Please be careful: switches that are VLAN capable often have a default VLAN configured. Do not choose the same VLAN ID as used in the switch or change the ID of the switch default VLAN. Parameter: Vlan_Inst_Count ------------------------------------ Type: integer Default value: None! You MUST set this value! Valid values: 1 up to the number of VLANs configured for the appropriate adapter. is a placeholder that has to be replaced with the actual instance number of the adapter given by the system. This parameter has to be set to enable the driver to check the completeness of all VLAN parameters you must set during configuration. Example: We use the sample entry from the beginning of this chapter to show the use of this parameter. Vlan_Inst0_Enable="Yes"; Vlan_0_DevNum=0; Vlan_0_ID=2; Vlan_0_Jumbo="Off"; Vlan_100_DevNum=100; Vlan_100_ID=4; Vlan_100_Jumbo="On"; Vlan_Inst0_Count=2; Two VLANs have been configured for this adapter. Therefore Vlan_Inst0_Count is set to 2. If you, for example, forget to set Vlan_100_ID, the driver will ouput the following message on the console: tcgeX: GetConfiguration: Missing VLAN parameter Vlan_100_ID! X is the instance number of the according adapter. After the display of this message, the driver will abort the complete configuration of the adapter because it makes no sense to continue. A possible workaround is to look up the configuration file and fill in the missing parameter. In case of two VLANs this might seem overstated, but if you have 10 or 20 VLANs configured for one adapter it can easily happen that you forget to set one parameter. ADDITIONAL PARAMETERS --------------------- There are additional parameters that can be set, but it is not necessary to use them. If you do not set these parameters, default values will be used by the driver. Parameter: Vlan__Jumbo ------------------------------ Type: string Default value: "Off" Valid values: "On" "Off" Example: Vlan_100_Jumbo="On"; If you want to use Jumbo frames on one or more VLANs you have configured for an adapter, set the appropriate parameter to "On". The adapter will be configured accordingly. NOTE: Do not forget to enable setting the MTU in /etc/rcS.d/S50_tcge! IMPORTANT: If you have configured your VLANs in tcge.conf manually and you want to use the script /usr/sbin/tcge_vlan_config the next time you have to change or add existing VLAN entries, please edit the file before using the script and add two extra lines for each instance: Insert # BEGIN VLAN configuration settings for instance in front of the first VLAN entry and # END VLAN configuration settings for instance after the last VLAN entry belonging to this instance number. is a placeholder for the instance number you have chosen. A VLAN entry for instance 0 would look like this: # BEGIN VLAN configuration settings for instance 0 Vlan_Inst0_Enable="Yes"; Vlan_0_DevNum=0; Vlan_0_ID=2; Vlan_0_Jumbo="Off"; Vlan_100_DevNum=100; Vlan_100_ID=4; Vlan_100_Jumbo="On"; Vlan_Inst0_Count=2; # END VLAN configuration settings for instance 0 NOTE: You have to do this for each VLAN instance! IMPORTANT: The /etc/hostname. files are important. One of these files is needed for each interface to be configured with 'ifconfig' by the system at boot time. You have to create one file for each VLAN interface you want to use. These files contain only one entry: the hostname or IP address associated with the VLAN belonging to the appropriate interface. If you put a hostname into the file you have to make sure that there is an entry with the correct IP address for this hostname in the file /etc/hosts! Example: For the two VLANs above you have to create two files: /etc/hostname.tcge0 /etc/hostname.tcge100 NOTE: If you want to use a different netmask as the default for the chosen IP address class for some or all of your interfaces at boot time, do not forget to put the correct value into /etc/netmasks! Now you should be able to configure your adapters for the use of VLANs. When the configuration is finished, you have two possibilities to activate the changes in tcge.conf: - Reboot the system. - Unload the driver with rem_drv and load it again with add_drv. For details refer to the description in section 4.1. *** 5 Tuning ========= This section describes settings that affect network performance. Also refer to the description of the CopyThreshold parameter above. The TCP/IP protocol stack of Solaris can be tuned to better suit high speed network adapters. This tuning is executed with the "ndd" tool. The startup script /etc/rcS.d/S50_tcge is created during installation and it sets some of the TCP parameters. More improvement can be achieved by further increasing e.g. tcp_xmit_hiwat and tcp_recv_hiwat when using Jumbo Frames. But this also increases the memory consumption per TCP stream and the effect depends on the type of application. If you need this last bit of optimization, please refer to the man page of 'ndd' and find the best settings for your system by experimenting with different settings. *** 6 "ndd" support ================ This driver supports the Solaris tool 'ndd' for reading driver parameters and statistics. For details on 'ndd', refer to the man page. The driver supports the following parameters: - ? : a list of all parameters is displayed. Do not forget to quote the question mark with a backspace: ndd /dev/tcgeX \? Otherwise the ? will be interpreted by the shell as a metacharacter. X in tcgeX is the number of the according interface. - link_status : Up or Down - link_mode : 1=full-duplex, 0=half-duplex - link_speed : 10|100|1000 (Mbits per second) - adapter_id : adapter identification string - hw_revision : hardware revision number - instance_num : instance number of adapter - ring_sizes : a list of the following parameters is displayed: - RX ring size: # of RX descriptor ring buffers - TX ring size: # of TX descriptor ring buffers - port_props : a list of the following parameters is displayed: - Link Status: see link_status above. - Link Speed: see link_speed above. - Jumbo Frames: jumbo frames enabled? - VLAN: VLAN support enabled? - RX ring size: # of RX descriptor ring buffers - TX ring size: # of TX descriptor ring buffers - AutoNegotiation: auto-negotiation used? - Duplex Mode: half- or full-duplex mode? - Flow Control: used mode - Role: Please refer to parameter Role_A_InstX in section 3.1 for details. - vlan_props : if VLAN support is enabled on the according interface, a list of three parameters (used in tcge.conf) is displayed: - VLAN Number: used for configuration purposes - VLAN ID: set by user - VLAN Jumbo Frames: jumbo frames enabled? If VLAN support is disabled, the following message is displayed: "No VLAN support on tcgeX enabled." X is the number of the corresponding interface. - vct_start : Start a VCT cable test on the selected interface. - vct_status : Get the actual VCT/DSP status of the selected interface. - vct_result : Get the result of the last VCT/DSP cable test of the selected interface. Please refer to the next section for details about the VCT parameters. Example: If you want to know the adapter identification string of the adapter belonging to IP interface tcge0, enter the command: ndd /dev/tcge0 adapter_id *** 7 Virtual Cable Tester(TM) (VCT) ================================= The Marvell(R) VCT technology utilizes Time Domain Reflectometry (TDR) technology to remotely diagnose the quality and characteristics of the attached cables. Using this technology it is possible to detect and report potential cabling issues such as cable opens, cable shorts or any impedance mismatches in the cable and accurately report - within one meter - the distance to the fault. The VCT technology enables the IT manager or the end user to quickly identify the failing mechanism and isolate the source of the problem. If at the selected device a link is up, DSP (Digital Signal Processor) is activated, which reports the length of the cable (only if link is at 1000 Mbps speed). VCT can be used with the Solaris tool 'ndd'. It is supported with the following three parameters: - vct_start : Start a VCT cable test on the selected device. - vct_status : Get the actual VCT/DSP status of the selected device. - vct_result : Get the result of the last VCT/DSP cable test of the selected device. Example: If you want to start a VCT test on device tcge0, enter the command: ndd /dev/tcge0 vct_start If you have started VCT on device tcge0, you will get the following message: "VCT test started on device tcge0!" VCT needs about three seconds to finish. If you try to get the results before finishing the test, you will get the following message: "VCT test is running on device tcge0" VCT can not be started if the link on the selected device is active. If you should do this accidentally, you will get the following message: "VCT test not started on device tcge0! Link is up!" If you have tried to start VCT on a device not supporting it, you will get the following message: "VCT Error: VCT not supported on device tcge0!" This message will also appear, if you try to view the VCT status or to get the VCT result on a device not supporting VCT. You can retrieve the VCT result with the following command: ndd /dev/tcge0 vct_result You will see output like the following example in the terminal window: Old DSP result for tcge0: Cable length: < 50m New VCT result for tcge0: Pair 1 [1-2] Length: 5m Status: Open in cable. New VCT result for tcge0: Pair 2 [3-6] Length: 5m Status: Open in cable. New VCT result for tcge0: Pair 3 [4-5] Length: 5m Status: Open in cable. New VCT result for tcge0: Pair 4 [7-8] Length: 5m Status: Open in cable. The first line belongs to the DSP output. The VCT result lists the cable pairs, the status of the corresponding cable pair, the distance to the fault (length), and the status of the test. "New" and "old" in the output above (and also in the output of the vct_status command, see below) have the following meaning: For VCT data, "new" means that the link is down and VCT has been executed during this link down time. "Old" means that the link is or has been up after the test has been executed, so the data is outdated and may be wrong now. For DSP data, "new" means that the link is up and DSP has calculated the cable length automatically. "Old" means that the link is or has been down after DSP has been run so the data is outdated and may be wrong now. The following states of a cable pair are possible after VCT has been executed: - Normal cable. : The cable pair is connected correctly. - Short in cable. : Two or more cable pairs are short-circuited together. VCT reports the distance to the short-circuit in meters. - Open in cable. : Lack of continuity between the pins at each end of the twisted-pair cable, i.e. the cable pair is not connected correctly. VCT reports the distance to the open location in meters. - Impedance mismatch. : The impedance on the cable pair is not correct. Possible reasons for impedance mismatch: - The cable pair is not connected properly. - The cable pair is damaged. - The connector is faulty. VCT reports the distance to the impedance mismatch in meters. - Test failed! : The test of the cable pairs was not successful. You can also retrieve the current VCT and DSP status with the following command: ndd /dev/tcge0 vct_status For the example above you will get the following message: "VCT status for tcge0: New VCT and old DSP data!" If the link was never up and VCT has not been started up to now, you will get the following message: "VCT status for tcge0: No VCT or DSP data!" *** 8 Troubleshooting ================== This section describes some common problems and their solution. Problem: The installation prints the following message: "Driver (tcge) successfully added to system but failed to attach" Solution: The driver file could not be executed. Check if you have the correct driver version for your system architecture. In the directory above the "sol3C940" subdirectory, enter: file sol3C940/reloc/kernel/drv/tcge This will tell you which system type the driver supports. Problem: The driver startup fails with the message: "tcge: BoardAllocMem: ddi_dma_mem_alloc failed!" Solution: is the instance number assigned to the adapter by the system. You probably have multiple adapters installed. Each one uses an amount of limited DMA memory. If the driver can not get sufficient memory, it will not work. You can increase the available DMA memory by tuning a kernel parameter. In the file /etc/system, add the line: set lomempages= The default for is 36. Increase this value until the above message disappears (recommended increment: 10). You have to reboot your system to activate this change! Problem: After installing additional adapters and reinstalling the driver, the network is up, you see the interfaces with 'netstat -i', but you can not reach other machines. Solution: The order in which instance numbers are assigned depends on the PCI slots the adapters are plugged into. This can cause an adapter that was inserted at a later time to have a lower instance number. Since you do not know this order when entering the IP addresses, the address assignment may be swapped. You can either swap the names of the corresponding /etc/hostname.tcgeX files or you can swap the cables of the adapters. X is the number of the corresponding interface. *** 9 Error messages ================= This section describes error messages that may be printed by the driver. This list is incomplete! If you get other error messages and you can not solve the problem, contact 3Com technical support. NOTE: In all following error messages, is a placeholder for the instance number assigned to the adapter by the system. Message: "tcge: Getconfiguration: Missing VLAN parameter !" Meaning: can be any name described as 'MUST be set' in section 4. You have configured the adapter tcge for the use of VLANs and forgotten to set a parameter that is essential for the correct functionality of a VLAN (Virtual LAN) device. If so, the attachment of the adapter is aborted. Message: "tcge: Getconfiguration: Illegal value for , using default!" Meaning: can be any parameter name described in section 3. You have entered an invalid value for this parameter in the driver configuration file. Message: "tcge: SkGeAttach: ddi_soft_state_zalloc failed!" Meaning: Not enough kernel memory is available for the driver. Message: "tcge: Adapter failed." Meaning: A serious driver or hardware problem has occurred. This will prevent the adapter from working correctly. You can check the adapter with the 3Com adapter hardware diagnostic program. Message: "tcge: Port A failed." Meaning: Same as the previous message, but for the port only. Message: "tcge: BoardAllocMem: ddi_dma_mem_alloc failed!" Meaning: The driver could not get enough DMA memory. See the section above for a solution. ***End of Readme File***