Blog

November 10 2017

Yet Another Explanation Of What Is Cloud Native

Cloud Native Software is software that gets its configuration information dynamically.TL;DR — Too long; didn’t readBackground, Defining Release Management vs Configuration Management.Release Management, This is what we’ve been doing with software sinc…

July 15 2016

Openstack all in one host (AIO), Virtualized install of Openstack with Fuel on KVM hypervisor

Jeffrey Gong

Virtualized Openstack single node installation with Fuel on Ubuntu KVMThis document contains instructions that will do a install of Openstack all on one host. This is a virtualized install of openstack with Openstack Fuel installer on a vm. …

April 30 2016

Getnet Plan Re-Published

Jeffrey Gong

Since my last post, I’ve taken a contract job in the bay area. I’ve had an active interest in Openstack since I’ve heard of it in 2011. It’s because I really needed an Openstack like support system when I was running an ISP.

The Ghetto Stack blog post wasn’t a compete serendipitous event. I had a job inquiry from placement firm that there may be another Openstack job opportunity a month prior. As I continued my vacation, I decided it was just time to power up some silicon again, while I sent my resume out twenty times a day. I really wanted to stay in Arizona, that wasn’t in the cards for me.

I’ve relocated to Sunnyvale California. I would expect in the year 2016, In the technology center of the WORLD, I should be able to get gigabit Internet connection to my over priced apartment. Despite proudly proclaiming to be in the heart of Silicon Valley, This city is no better off with regards to connectivity than any other American metropolitan area.

I’ve never fully appreciated all the business forces that affect our lives, until I look time off and went to business school. More than anything else, It was a good time, with good people, and an opportunity to rethink technology without constraints of being an technology person.

A tangible out put from that experience was a way to organize a complex idea in to document that explains how to have cost effective communications competition to every home. Because this idea is seemingly too good to be true, There is an equal and opposite opposing force. Cost effective competition means no one company can ever benefit from a monopoly status. The logical follow on, is that no company will ever want to support this plan. In fact, the Incumbent will viciously fight this plan.

The local cable company here in silicon valley has really bad reviews. I signed up anyway. Same story, more bandwidth, lower cost. I’ll deal with the over worked Comcast employees stuck in a ill-conditioned business that has no chance of survival in a competitive environment. After several bad experiences, on line and over the phone, I canceled it. I found Sonic Internet. I know their struggle well, I had the privilege to lived it. As I wait for my sub-speed DSL to be delivered, I’ve decided to come in the office on a Saturday to get an Internet connection. I’m republishing the post I did in 2009.

I’ve been here many times before. Comcast will shrug off and counter all clams of poor customer service with were big and these are just “isolated incidents”. But, If you believe that they are fat and happy, and just to big and slow to provide competitive service, just maybe, Comcast is vulnerable.

The other force fighting this idea is apathy. What we have is good enough, I’m more interested in other things.

I understand that local city councilmen probably will not fully grok this idea in one read. That means that we the consumers need to motivate a majority of them to understand this, and take the risk of reallocating the cost of deploying fiber to the owners of the streets. Communications is a long lived commodity infrastructure that is as necessary as roads, water, electricity, trash, and sewage. The costs of fiber to the home should be put in the same bucket.

Prior mis-understanding of this idea is that local City Governments will be your Internet Service Provider. City Governments will not be your ISP. Local City Government will do what they do best. They will be the body the oversees ownership of the fiber in the ground and designate some organization to coordinate fair competitive access to the fiber for all interested parties.

I’m told buy Sonic, I’m going to pay $10/month in taxes to the city. Why should I pay that much in taxes for wire in the ground that was placed there before they were born? Say it, extra ordinary profits. If I’m paying $10/month in taxes, I should have fiber to the home! We as consumers should demand fair value for the fees we pay in taxes and to ISPs.

Consumers will only get communications choice when they demand it from their local city governments.

I now reintroduce the Getnet Plan.

Jeffrey Gong — Sat Apr 30 11:42:52 PDT 2016

The Getnet Plan

White Paper

On use of

Passive Coarse Wave Division Multiplexing

For

Promotion of Competition at the Last Mile

January 8, 2009

Jeffrey Gong

Overview

Most consumers don’t have fiber optic communications to the home. It is a very expensive and risky investment for more than one or two providers to install fiber optic communications to all addresses. Consumers (and the economy as a whole) benefit when there is broader competition to provide services. A solution is to allocate responsibility for installation and management of communications transport to municipalities using Coarse Wave Division Multiplexing (CWDM) to promote competition between multiple service companies to the same address.

Justification for The Plan

Fiber optic cable is the preferred communications transport. No other transport cable comes close with regards to capacity, distance, and low costs. At some point in the future, fiber optic cable will be installed to every business and home. Consumers will eventually pay for the cost of the network plus a profit for all parties that participate in its operations. Paradoxically, these cables will be in city right of ways (ROW) and we, the taxpayers, own the ROW.s. We as consumers want choices and competition. It makes little sense to have multiple carriers run parallel cables to the same properties. The same fiber will work for cable TV, phone and internet because it can carry multiple services on the same fiber. Fiber optic cables are nothing more than dumb pipes.

A solution that meets the above criteria can be achieved if fiber optic communications transport are installed, owned and managed as a city service. Consumers and service providers would be able to order connections as needed. Cable TV, internet connectivity, and phone services would all be able to use this common transport service at the same time. Installations and maintence will be aligned with other high cost commodity services like roads, water, and sewage.

The following are eight reasons to support this plan. The technology to implement this plan is available now. All we need is the vision and the will to execute this plan.

Reason 1: Now is the Time

Fiber to the home (FTTH) is not widespread.but rapidly becoming so–which allows a decision to be made now avoiding potential regulatory and political issues. Capital expenditure to create a new fiber network to all homes can be made now without significant obstacles if a private entity gets to the table first.

Traditionally, risk-adverse municipalities would want a publicly traded business to bear all of this responsibility. In practice, this could be a potential liability if these companies fail. In bankruptcy, the shareholders will get wiped out, a new buyer will assume the assets in the streets, and the consumers will again pay for the misdeeds of the failing corporation. Separation of these companies from the physical wire in the ground is a natural way to isolate the risk of competitive access carriers. By doing this now before the costs have been incurred to deploy the fiber, our grandchildren will be spared the burden of super normal profits to the owner of the fiber network. There are twisted pair phone wires in the ground that have been generating monthly revenues for the phone company for more than 100 years.

In this communications plan, when a carrier fails, customers could just switch their connection to a new carrier. All the legal issues associated with ownership of wires in the street are gone. All issues associated with a company too critical to fail are gone. Any single service company failure will only affect a portion of the community. Poor service will no longer be rewarded with rate increases. This plan will isolate risks associated with poor management of a communications company. The critical wire infrastructure will no longer be used as leverage to price discriminate.

The full cost of installing fiber to every address has yet to be incurred. It is in the best interest of the community to take the initiative to put a fiber optic network into place. It should be done so in a manner that enables competition. Setup in this way, it can be done in a method that gives ownership to the citizens of the communities it services. It may be managed by the same organization that manages other fixed cost infrastructure like roads and water lines.

Reason 2: Fiber Optical Cable is a Vital City Service

The use of communications has evolved from an optional value added service to become a vital service. A precedent for this recognition may be seen with the Communications Act of 1934 that created the Federal Communications Commission (FCC). The Universal Service Fund was founded to advance the availability of low income, rural, insular and high cost areas at reasonably comparable costs. The Universal Service Fund was funded internally by the Bell monopoly. These actions clearly recognized the importance of communications and put it into law.

Costs for fiber optical cable, CWDM multipliers, and single mode transceiver are all inexpensive commodity products at this time. This is very similar to commodity pricing for the products that are used to construct roads, water, and sewer services. All of these services require access to each home, are capital intensive to install, and have long expected use lifespan. Regardless of the cost of installation, it is a one-time expense. It may be financed and paid back over time like all other city capital expenditures. The match is undeniably similar. We as residents of a city collectively own our roads and water lines. It just makes sense to make the choice to own our fiber as well before costs are incurred.

Investments in fiber optic infrastructure have the same potential to generate revenues for the new owners for decades, if not centuries, into the future. It would be in the best interests of the consumer to let our children own the wires that we will pay to build. In the past, it made sense to let a monopoly own and run a network to deploy an application specific network. The past business case did not allow for alternate uses. Because technology allows multiple generic uses and competition, it makes sense to change policies to reflect current status of technology.

The precedent for enabling a government organization to perform a service on behalf the community is seen in the postal service. Allowing neutral government control of last mile fiber transport is no different. In a time where legislatures struggle with declining tax revenues from phone services, it makes sense to shift the focus from regulation of phone service to regulation of transport on fiber optical cables. We are advocating modern policies to reflect the current state of technology and current needs of the community.

Reason 3: Alignment of Ownership and Management Interests

One of the roles that a city manager will perform is to provide fair use of publicly owned roadways. A competing goal of all private utilities is to gain access to all locations that are economically beneficial for their use. By shifting the responsibility of all communications cabling to the city, we are aligning the use of the street with management of space in public streets.

The .right to exist. in a roadway now becomes a new administrative process that needs to be created and managed. This is another source of administrative overhead that we as taxpayers will bear as increased operational costs for a city management.

If we remove competitive fiber optic carriers from having access to the streets, we will not have to repeatedly tear up the street every time a new fiber optic based services provider wants access to the same road. It makes little sense for city managers to allow roads to be torn up every few years. It makes expensive well-paved road look and drive like test tracks.

In a system where there are multiple fiber optic network owners competing for the same space on the roadway as other services like water, sewage, electrical power, etc., things will go wrong. It is the duty of management to define how these failures are handled. It goes beyond an accidental cut with a backhoe. It can progress into outright theft of someone else.s conduit. If someone sees an unused conduit and they place services in it, we now have to deal with conflicts and potential litigation.

The city owns the streets. They also own water and sewer line in the roads. It is a natural alignment of responsibility to have the city own the fiber in the ground. Improving operational efficiency in city operations and in removing multiple competitive access carriers installing fiber in the street translates into real operational cost savings. These savings will be passed onto the consumers.

Adopting this plan will align the interests of fiber ownership with the interests of management of the streets and management of fiber cable. This plan will have operational efficiency that will be passed on to consumers. Operational efficiency in this case is beneficial because savings come from reduced administrative overhead.

Reason 4: Passive CWDM Technology Enables Plan

Coarse wavelength division multiplexing (CWDM) is a means of combining and/or separating multiple signals of different laser wavelengths into or out of a fiber optic transmission cable in a near passive network. This is exciting because it uses no electrical power to operate.

The alternative today is the use of active electronics. This network does the same job but is significantly more expensive to operate due to its use of power. It uses rack-mountable pieces of electronic equipment that may cost $50,000 per node. The current method of deploying critical communications equipment is to install it with an uninterrupted power supply, air conditioning, and optional back-up power generation. You also need to keep spares parts or service contracts and personnel on staff to operate it all.

Passive CWDM requires almost none of the above. CWDM is a prism that does the same job of combining and splitting different wavelengths onto one fiber. There is no electrical power to operate it. There is no electronic equipment with configuration parameters to set and change. That means there is no engineer to manage it. No need for electricity means there is no need for an uninterrupted power supply, back-up diesel generators, air conditioning, and service contracts to keep it powered and running. It is cheap in comparison to the active alternative.

Each wavelength on a fiber optical cable easily supports current applications. It.s safe to say that the majority of the fiber that is currently deployed is using only one wavelength, most likely 1310-nm. A passive CWDM currently costs a few hundred dollars per unit. I predict that it will get as cheap as a few dollars per unit when they are widely accepted and produced in volume.

Reason 5: Fiber Cable is a Dumb Pipe

Any communications transport medium, twisted pair wire, coaxial, or fiber optic is defined by the equipment on both ends. The actual wire has no intelligence. In ideal cases, the transport requires no power. All of the aforementioned transportation mediums have limits. If the distance to be covered exceeds the respective limits, it must have one or more repeaters to boost the signal. This is the most common need for electrical power. To run data communications over any of the above three mediums, matching equipment on both ends is needed with correct medium adaptors. When this is done correctly, a network connection will be constructed.

Because fiber is just a dumb pipe, it should be treated as such. Since the beginning of communications with wire technologies, companies have earned a price premium with regards to the cost of network deployments because there was no alternate use of the wire. Fiber optic cable is the first transport that can replace all current copper communications. The expected use may now be expanded to include multiple carriers doing the same service.

Dumb pipe means that the cities that assume operations of a fiber network will have a limited range of maintenance requirements. Either the cable will carry a laser or it will fail. If it fails, the operators would use an alternate pair or find the cable break and repair it. This is a task that is well defined. It can be managed on a large scale with government grade employees.

There is no magic in the cable. The magic happens in the equipment on the ends of the pipe. Armed with this knowledge, we should adopt a system where there is cost effective fair access to the cable.

Reason 6: Enables Competition

Laissez-faire works great in a system where goods and services are easily exchanged. Near-monopoly control of systems by telcos and CATV providers does not allow for freer competition. The United States Congress passed the Telecommunications Act of 1996 that forced incumbent local exchange carriers (ILEC) to share existing phone lines with competitors. This act of congress has not created a viable ecosystem of competitive local exchange carriers (CLEC). A fiber optical communications network that allows competition will only happen if we design a publicly owned system to support it.

There are currently 17 defined wavelengths in CWDM with 20nm spacing. Dense wave division multiplexing (DWDM) uses the same spectrum with 2nm spacing. The actual limit of how thin the spacing can be sliced is not known. There is a lot of room for growth. By allowing open market access to these wavelengths to all premises, we will have effectively created a system of market competition with no regards to what type of service that may run on a given wavelength. This is analogous to assigning service providers to different channels on the same fiber.

Single Mode Fiber is the preferred technology to be used in city streets or long haul networks. It has a stated range of 50km (31 miles). Many cable vendors state a range of 70km (43 miles). This means that a single wire center can have all the needed connection in a 30-80 mile diameter go to one location with no need for repeaters. Actual fiber deployments will vary according to specific real estate topography and political considerations.

Networks work best when there is one administrative body for a given network. By allocation of just transport responsibility for the local loop to one organization, we will have the best chance for efficient network deployment, management, and fair competition.

Reason 7: Promotes Innovation

Each wavelength currently can transport 2.5 Gigabits per second for a wavelength. This will yield a 1Gbps bi-directional connection. Single mode gigabit transceivers sell for about $500.00 new. The leading edge of technology is 10Gbps for a frequency is available off the shelf. However, these speeds are not available until FTTH is deployed.

Existing services like cable TV, telephone, and internet all use much less bandwidth than any single wavelength can carry. The goal of having one service organization that can provide TV, phone and Internet is called a triple play. That means there is room for 17 different triple play providers to all homes with fiber.

Because the actual wavelength is mapped from end point to end point, the choice of line protocol is up the consumer and service provider to choose. This open access to the lowest layer combined with the large bandwidths will maximize opportunity for innovations using existing or yet to be invented protocols.

Reason 8: Will Keep the USA Globally Competitive

Like a personality, all countries retain a different global competitive advantage.

The USA has been a market leader in technology. The irony is that a free market economy is at a disadvantage in network construction and operations. Networks are best run and operated by a single organization. By supporting this idea of reallocating responsibility of just the communications transport to neutral party, any nation will become or remain globally competitive.

It is my opinion that the USA must act on this plan to maintain a competitive posture. There are other countries that have nationalized fiber infrastructure to all addresses. This means that the USA is now at a competitive communications disadvantage with these nations. This is done because we continue allow a handful of companies to use their monopoly status to earn super normal profits.

Closing Remarks

There is a lack of fiber optic communication to the home, a lack of real competition of services to the home, and there is significant capital risk for such a project. We have presented a two-step process for solving all three of these issues. The first step is to allocate the responsibility to build a fiber network to local cities that already have experience in exposure to these risk factors. The second step is to use CWDM to allow multiple competing companies parallel access to the same address.

Historically proven importance of communications is impressive. Looking forward, expanded use of communications may be an alternative to travel. This is important given the current awareness of personal carbon footprints. Other benefits include creating a communications system that keeps our country globally competitive.

March 3 2016

Jeffrey Gong 2016-03-03 19:44:00

Jeffrey Gong

Ghetto Stack

Install of Openstack with Fuel, Dell R710 and Cisco catalyst 3548. Originally done with fuel 7. This install will work with fuel 7 through 10

Summary

Hardware used in this article is outdated by today’s standards. In addition, the networking is a minimal stick something in place to make it just work. This exact configuration is meant to demonstrate what is a minimal networking needed to make an install work. Hence the Ghetto Stack theme. This is meant to be a reference. Since this post was done, I’ve been able to replicate this pattern to use Fuel to install Openstack all in one host.

The goal is to use Fuel 7.0 defaults as much as possible. This is a write up about what happened when doing an Openstack deployment to physical hosts. This is a convention over configuration approach. This configuration will work with Fuel 6, 7, 8 and 9.

Dell R710 servers are obtainable in used condition at a relatively cheap price. The suggested target price is about $250 each at this time. Each server should be a working configuration that has 2 x quad core CPU, Memory, and Hard drives, DVD, and two power supplies. Prior generation servers are being recycled. For servers in this price range, operational costs like shipping, electrical, labor and other operational costs will exceed the value of the hardware. This once again contributes to the Ghetto Stack theme.

This configuration requires a managed Ethernet switch. Individual ports of the switch need to be configured for different networks. A used 3548 was brought back from its retirement in a garage for this project.

We use commodity NAT routers to provide Internet accessible IP address. This is a simple, easily understandable, and cheap way to achieve this task. It also maintains the keep-it-simple, ghetto theme. There will be a total of three instances where we apply NAT in Ghetto Stack.

This NAT router configuration does not allow traffic from one private segment to other private segments. To work around this, the workstation used to configure, test, and run Openstack is configured to have an IP address in three different network segments.

Reading this document will provide the reader with an understanding of how to configure a network to host OpenStack on physical hosts. Openstack is written in Python, and most Python code acknowledges a convention over configuration approach. That means there are some reasonable defaults chosen. The Openstack community has chosen to have a very minimal number of defaults. Fuel installer goes father and has chosen enough defaults to make Openstack actually work.

This setup will work for Openstack Fuel 8 and Fuel 9.

All the host systems used in this setup had two, four core processors, 24 gigs of ram. Network connection was 100 MB switch. The install ran to completion, but the resulting stack has issues. The Fuel all in one host install and runs with a 96 GB ram host. About 20-30 Gigs of ram is consumed for paging and caching. It’s my opinion it’s not the best use of time to do a multi host install of Openstack with each node less than 64 Gigs of ram.

Design Description

IP Network Description

Home Network is the first NAT routing device entering the metaphorical building. It’s the gateway to our Internet provider. In this example, its IP address will be 192.168.0.1/24. We will construct a Openstack deployment that will leave this network in place. In this case, altering this network will cause too much support work. This segment will be isolated by assigning it to VLAN 2.

Fuel Network will be built on 10.20.0.0/24 address space. Default IP address for the Fuel server is 10.20.0.2/24. The Fuel server does it’s own DHCP, BOOTP, and DNS for this network. It also assumes this address space is internet accessible. Default fuel setup assumes that there will be VLAN (802.1Q) encapsulated packets on the same segment. That means we will assign 10.20.0.0/24 to VLAN 1. Switch ports will be configured with “switchport trunk encapsulation dot1q”. The magic of this configuration is that un-tagged packets will be put on VLAN 1. DHCP and BOOTP will use this feature. Tagged traffic will also be switched to the correct ports. It’s assumed that 10.20.0.0/24 has Internet access. This configuration used the sole desktop Workstation for this task. If you want Openstack to stay running while the desktop is being rebooted, use another NAT router instead of the workstation. It’s OK to think it, Ghetto Stack!

Public Network is on 176.16.0.0/24. by default. Fuel expects this to be on a separate interface, on each Openstack node host. Traffic is not VLAN tagged by default. The Ethernet switch port connecting to the public port on each host was connected to switch with the port tagged to VLAN 100. Cisco Lan environment calls this VLAN Native. The packets coming from each of the nodes is not VLAN Tagged. To map this to our setup, each port connected to the nodes has the configuration of “switchport access VLAN 100”. It was only done this way because VLAN tagging is not selected by default on the configuration screen. Again, this is the only other segment that needs to have an Internet accessible IP address. Another commodity NAT router may be used here. In a production deployment, we would use IP addresses assigned by our network administrator or ISP.

Management Network and Storage Network are by default out on VLAN 101 with 192.168.1.0/24 and vlan 102 with 192.168.1.0/24 respectively. Through the magic of a correctly configured VLAN, these networks work.

Network Diagram

VLAN Network Description

VLAN 1 – default, fuel DHCP, all eth0 from servers
VLAN 2 – existing Home Network
VLAN 100 – Public IP address.
All other VLAN just magically work

Switch Configuration.

set password and enable password
set IP address of switch to 10.20.0.254/24
put ports 1 through 8 on VLAN 2
put ports 9 through 32 on VLAN 1. port configured to “switchport trunk encapsulation dot1q”
put ports 33 through 48 on VLAN 100
save configuration

Physical Network Description – Ethernet wiring

Fuel Server

eth0 on to VLAN 1 (switch port 9)

All Openstack nodes

eth0 on any switch port 10 through 32, defaults VLAN 1
eth1 on any switch port 10 through 32, Not required Option
eth2 on any switch ports 33 through 48, tagged to VLAN 100
eth3 on any switch ports 2 through 8, IPMI and Home Network DHCP IP address for debugging

NAT Router 2

Public, or Wide Area side connected to VLAN 2
LAN or Private side connect to VLAN 100
Set IP address range to 176.16.0.1/24, no DHCP

Desktop Workstation – Ubuntu host, NAT Router

eth0 connected to Home Network. If you connect it to a VLAN 2 port, you will have to move your Ethernet wire if you choose to shutdown the switch.
eth1 connected to a port on VLAN 1. IP Address set to 10.20.0.1/24
Use the following commands to provide NAT translations for 10.20.0.0/24 network.
If you want Openstack to run while your desktop is being rebooted, use another NAT router instead of your workstation. Make NAT Router 10.20.0.1, no DHCP. I would suggest that the Workstation should use 10.20.0.254/24 as an IP address in this case.
eth1.100 127.16.0.128/24
this is VLAN tagged. Allows access 127.16.0.0/24 for validating this subnet. IP Address not used by default by Fuel.

There will be may variations for Workstation Network configuration. In the event you only have one ethernet port, consider using VLAN tagged interfaces for any or all of the networks.

Putting it to Together

Powering on the stack.

Standard disclaimer requires me to say, if you have to ask, you need to consult an electrician.

With that said, I couldn’t find my amp meter until after I bought a new one. Uggh… Most modern homes in the USA have 120 V outlets available in the home. It would be common to find 14 gauge wires with 15 AMP breakers. If you have this setup, it doesn’t mean you can draw a full sustained 15 amps through this circuit indefinitely. You should target half that amount. If you go above that do so at your own risk. You should have a goal of never having the circuit breaker kick off the power.

Initial setup, one Dell 2950, seven Dell R170 nodes, one 3548 switch, and one monitor. All plugged into a Tripp-Lite iso bar power strip with a 15 amp circuit breaker. T amp meter showed 12.1 amps immediately after being powered on. Running all the servers at the same time for a few hours kicked the 15 amp circuit breaker in the power strip. I have not run all the servers through one breaker since this time.

Network configuration of Ubuntu 14.04 workstation

Below is an interface file from desktop. By connecting one Ethernet port to the Home Network, and the other for Fuel deployment, the workstation will work as expected with Ghetto Stack powered off.

# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
  address 192.168.0.9
  netmask 255.255.255.0
  network 192.168.0.0
  broadcast 192.168.1.255
  gateway 192.168.0.1
  dns-nameservers 192.168.0.1
  dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet static
  address 10.20.0.1
  netmask 255.255.255.0
  vlan-raw-device eth1

auto eth1.100
iface eth1.100 inet static
  address 127.16.0.128
  netmask 255.255.255.0
  vlan-raw-device eth1

command line alternative:


modprobe 8021q
apt-get install vlan
vconfig add eth1 100  
ifconfig eth1 10.20.0.1 netmask 255.255.255.0
ifconfig eth1.100  127.16.0.128 netmask 255.255.255.0

This is an untested interface file for a workstation with one Ethernet port.

# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
  address 10.20.0.1
  netmask 255.255.255.0
  vlan-raw-device eth0

auto eth0.2
iface eth0.2 inet static
  address 192.168.0.9
  netmask 255.255.255.0
  network 192.168.0.0
  broadcast 192.168.1.255
  gateway 192.168.0.1
  dns-nameservers 192.168.0.1
  dns-nameservers 8.8.8.8

  vlan-raw-device eth0
auto eth0.100 iface eth0.100 inet static   address 127.16.0.128   netmask 255.255.255.0   vlan-raw-device eth0 


command line alternative:

modprobe 8021q
apt-get install vlan
vconfig add eth0 2  


vconfig add eth0 100  
ifconfig eth0 10.20.0.1 netmask 255.255.255.0

ifconfig eth0.2  192.168.0.9 netmask 255.255.255.0
ifconfig eth0.100  127.16.0.128 netmask 255.255.255.0


NAT 10.20.0.0/24 network on Workstation


echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables --flush
/sbin/iptables --table nat --flush
/sbin/iptables --delete-chain
/sbin/iptables -P FORWARD ACCEPT
/sbin/iptables --table nat -A POSTROUTING -o eth0 -j MASQUERADE

Configuration the switch

In order to get in through the console, I obtained a USB to serial cable and connected it to my workstation. I already owned a console cable. Connect the serial cable to the console cable, then the console cable to the switch.

Getting in to console

Next, do the following to connect to the switch:


sudo apt-get install cu
sudo chmod 666 /dev/ttyUSB0
sudo cu -l /dev/ttyUSB0 -s 9600

To disconnect from cu, type tilde dot

~ .

If you need to, do Cisco 3548 password recovery,

entering and exiting configuration mode


c3548#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
c3548(config)#end
c3548#

saving configuration


wr mem

undoing a configuration, just put a “no” in front of the line you wish to delete. Use the following session as an example.

c3548#show running-config interface FastEthernet0/33
Building configuration...

Current configuration:
!
interface FastEthernet0/33
 switchport access vlan 100
end

c3548#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
c3548(config)#interface FastEthernet0/33
c3548(config-if)#no switchport access vlan 100
c3548(config-if)#end
c3548#show running-config interface FastEthernet0/33
Building configuration...

Current configuration:
!
interface FastEthernet0/33
end

c3548#

Set the secret password. Use the following session as an example.

c3548#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
c3548config)#enable secret changeme
c3548(config)#end

Set the line password and secret password. If you don’t, you are not able to telnet in and make changes. Change the “line vty” passwords

line vty 0 4
 password YOURPASSWORD
 login
line vty 5 15
 password YOURPASSWORD
 login

Below is the configuration used.

interface FastEthernet0/1 was used to connect to another switch. There is no need to sit in the same room with noisy fans.
interface VLAN1 is set to 10.20.0.254, this is the IP address we can telnet to the switch.

interface FastEthernet0/1 switchport trunk encapsulation dot1q switchport mode trunk ! interface FastEthernet0/2 switchport access vlan 2 ! interface FastEthernet0/3 switchport access vlan 2 ! interface FastEthernet0/4 switchport access vlan 2 ! interface FastEthernet0/5 switchport access vlan 2 ! interface FastEthernet0/6 switchport access vlan 2 ! interface FastEthernet0/7 switchport access vlan 2 ! interface FastEthernet0/8 switchport access vlan 2 ! interface FastEthernet0/9 switchport trunk encapsulation dot1q ! interface FastEthernet0/10 switchport trunk encapsulation dot1q ! interface FastEthernet0/11 switchport trunk encapsulation dot1q ! interface FastEthernet0/12 switchport trunk encapsulation dot1q ! interface FastEthernet0/13 switchport trunk encapsulation dot1q ! interface FastEthernet0/14 switchport trunk encapsulation dot1q ! interface FastEthernet0/15 switchport trunk encapsulation dot1q ! interface FastEthernet0/16 switchport trunk encapsulation dot1q ! interface FastEthernet0/17 switchport trunk encapsulation dot1q ! interface FastEthernet0/18 switchport trunk encapsulation dot1q ! interface FastEthernet0/19 switchport trunk encapsulation dot1q ! interface FastEthernet0/20 switchport trunk encapsulation dot1q ! interface FastEthernet0/21 switchport trunk encapsulation dot1q ! interface FastEthernet0/22 switchport trunk encapsulation dot1q ! interface FastEthernet0/23 switchport trunk encapsulation dot1q ! interface FastEthernet0/24 switchport trunk encapsulation dot1q ! interface FastEthernet0/25 switchport trunk encapsulation dot1q ! interface FastEthernet0/26 switchport trunk encapsulation dot1q ! interface FastEthernet0/27 switchport trunk encapsulation dot1q ! interface FastEthernet0/28 switchport trunk encapsulation dot1q ! interface FastEthernet0/29 switchport trunk encapsulation dot1q ! interface FastEthernet0/30 switchport trunk encapsulation dot1q ! interface FastEthernet0/31 switchport trunk encapsulation dot1q ! interface FastEthernet0/32 switchport trunk encapsulation dot1q ! interface FastEthernet0/33 switchport access vlan 100 ! interface FastEthernet0/34 switchport access vlan 100 ! interface FastEthernet0/35 switchport access vlan 100 ! interface FastEthernet0/36 switchport access vlan 100 ! interface FastEthernet0/37 switchport access vlan 100 ! interface FastEthernet0/38 switchport access vlan 100 ! interface FastEthernet0/39 switchport access vlan 100 ! interface FastEthernet0/40 switchport access vlan 100 ! interface FastEthernet0/41 ! interface FastEthernet0/42 switchport access vlan 100 ! interface FastEthernet0/43 switchport access vlan 100 ! interface FastEthernet0/44 switchport access vlan 100 ! interface FastEthernet0/45 switchport access vlan 100 ! interface FastEthernet0/46 switchport access vlan 100 ! interface FastEthernet0/47 switchport access vlan 100 ! interface FastEthernet0/48 switchport access vlan 100 ! interface GigabitEthernet0/1 ! interface GigabitEthernet0/2 ! interface VLAN1 ip address 10.20.0.254 255.255.255.0 no ip directed-broadcast no ip route-cache

Installing Fuel

Fuel takes over the whole server. Download Fuel 7.0 ISO (torrent), burn it to DVD or copy to USB, Boot, and install it.

Do the update

configuring Fuel

enter, down arrow to Quit Setup, enter, right arrow, down arrow to Save and Quit, enter

It will now complete the install. 
The root password will also work for all nodes. Fuel sever installs ssh keys
on all installed nodes. From fuel server, you can ssh node-x, i.e.
ssh node-1.

There is an alert message that takes you to the flowing url, then to the next url to do an update. We are just going to do the update now.
https://docs.mirantis.com/openstack/fuel/fuel-7.0/release-notes.html#maintenance-updates
https://docs.mirantis.com/openstack/fuel/fuel-7.0/maintenance-updates.html#mu-v7-0
# ssh root@10.20.0.2
use password r00tme
do the following:
# dockerctl backup
# yum update
# docker load -i /var/www/nailgun/docker/images/fuel-images.tar
# dockerctl destroy all
# dockerctl start all
# fuel release --sync-deployment-tasks --dir /etc/puppet/
use web browser, go to url http://10.20.0.2

click on connect latter

select New OpenStack Environment

Name Environment one, click Next

select KVM radio button, click Next

select Neutron with VLAN segmentation, click Next

select No, use default providers, click Next

select Install Ceilometer, click Next

click Create

we new have an Environment with nothing configured. The Fuel server is ready. Now power on the nodes.

Configure Each Node

Go in to the bios by pressing F2 during boot, set eth0 to pxe boot as the first item in boot sequence. Fuel server will remember if a host has already been installed, and tell it to boot off of it's hard drive if so. Reboot server. It will pxe boot. It will take several minutes.

Upper right of this screen shows 6 over a 6. The top number is the number of hosts not allocated, and the bottom number shows total hosts. The bell with Red 6 provides more details.
click on Add nodes

select controller, Operating system, one available node, click Apply Changes

click Add Nodes

select Compute, Storage, Operating System, Select All, click Apply Changes

select, Select All, click Configure Interfaces

Left click on Public, Drag and Drop on to eth2
Left click on Storage, Drag and Drop on eth1
Left click on Private, Drag and Drop on eth1
Click on Apply
Refresh the page

Click on Networks Tab

Under the Management section,
change CIDR to 192.168.3.0/24
change VLAN tagging to 103
click on Save Settings

click on Verify Networks

on Verification succeed,
click on Save Setting
click on the Dashboard tab

click on Deploy Changes

click on Deploy
This install took 1 hr 17 minutes.

click on Proceed to Horizon
Tell me a story
I’ve had the privilege of being in an environment with lots of computers, customers, and home brew software. It is in this context, the term Technical Debit acquires a very deep and concise meaning to me. Openstack has a very noble goal, which is to provide an API that controls compute, storage and networking. It’s a very simple idea. The simplest ideas, when done right, are the most difficult. From this perspective, Openstack is by definition the compute, storage and networking version of Structured Query Language or SQL. SQL is the means in which we learned control data. When we didn’t have to deal with calculating, referencing, or translating data, we got better at managing data. Openstack is doing the same right now for compute, storage, and networking.
I’ve also had the privilege of contracting for many years. When entering a new environment, newbie have been the most successful when someone puts the effort in to introducing them to the ways of the tribe. I’ve since found out that the term tribal knowledge came from military personnel working with indigenous warring fractions. Business is the peace time equivalent of war. A reason Openstack is popular is because it lessons the Technical Debit burden for companies who view management of compute, storage, and networking as a non-core business. Companies that make it their core business to provide compute, storage, and networking services and solutions are now in a life and death proposition of competing with open source. It’s obvious from that perspective why all these big name vendors joined Openstack foundation. They and many new startups are jockeying for key positions in the value chain. This is Nash Equilibrium swirling in front of us right now. Applying the Model View Control pattern to view Openstack Marketplace is a great beer drinking topic, and for attracting women.
In the spirit of open source, I present the above article. A goal for me is to work on a cloud application without vendor lock in. I’ve installed and tried Devstack, RDO, Openstack Ansible, and Kolla. It’s time to see what a physical deployment looks like in comparison.
I had a Dell 1955 chassis with 7 blades. I wanted to redeploy that beast. The combination of needing 240V and space was a challenge. I was OK with fixing DHCP on request. 240V, and the 10 Gauge extension cord tapped in to the stove outlet or dryer outlet; snaking through the home seemed a bit too much. I’ve never met the man, but I bet Steve Jobs would have just done it.

The next consideration was to use the same generation of hardware with separate boxes. I had one Dell 2950. I could scavenge stuff from the Dell 1955 chassis. The litmus test I used to determine obsolescence of computer hardware was for comparing the value of hardware verses the cost of electricity to keep it running for a year. Intuitively, we know that all operators have a similar threshold. This is where Nash is working against me here. All the scrap yards have been sending that stuff to metal recycle bins for a while now.

At the end of the first day, I had no success using this generation of hardware. I gave in around noon on the second day. It was apparent that Clovertown was not the path of least resistance. I would have to let go of some green backs and go to Nehalem. I already had two R710. I bought one 3 years ago and one a few months ago. The cost difference is best measured in multipliers. I located a local suppler and decided to get 5 more. The goal was to get Openstack working sooner than later. I had planned for more hardware challenges by over-buying, and arranged a purchase the next day. The seller agreed to meet me at 7 pm at a convenience store. A cash deal took place in the parking lot on a Friday night. I’ve never done that before.

Saturday morning, I stacked the servers up in the spare bedroom. I placed a sign on the door, and now it is my server room. Placement of the nodes was towards the back wall. I left space all the way around the servers. I knew it was going to get hot in that room, also that it was prone to be a cabling mess.

Years before, I was advised by a lawyer I had once hired who said, “if you can explain what you did to a judge, and he doesn’t get a red face, it’s probably OK to do it”. I had visions of standing before a judge and saying, “Yes, the house burnt down because of an electrical fire” , “No, I didn’t check the amperage usage”, and “Yes, I have a degree in engineering.” In my imagination, he would then proceed to say, “Guilty of Negligence! Next!”

I drove to the nearest Home Depot to purchase an amp meter, feeling slightly amazed how much my project was costing me. This project was cutting into my beer budget. I consoled myself by thinking, “beer from the grocery store is just as good as the beer at a pub.”

I hooked up the amp meter then turned on all the servers. I confirmed that the amperage draw was about 80% of the 15 amp breaker feeding that room. My imagination wandered back to my hypothetical court case where I’d say, “I have no reason to believe that 500 pounds of computers, all hooked up to one outlet was a fire hazard, Your Honor.”

I started work on the Fuel server, which is Centos install. This installs itself very well by using Docker to host micro services. This is very cool in my opinion. I left the default 10.20.0.2 on eth0. I assumed it needed access to the Net, so I added a second IP address to the unused port of the fuel server and connected it to the Home Network. I gave that port a default gateway, and removed the default gateway from the other port. As suspected, it phoned home and got more stuff before it completed the install. Upon completion, there was a message on the console with defaults, showing the URL for admin screen, password and root password. Nice! No Google-ing for the details were necessary.

This project was pretty much all hacking. I glanced at the docs. Something in the back of my mind told me that this was not what I wanted to read. This was going to be a project where, if all else fails, read the docs. After getting a few nodes to DHCP boot, I discovered the Network Validate button. Even after NAT Translating 10.20.0.0/24 was confirmed to work, it still failed the validation test. Yes, I did try a few install runs with a failed validation. I can confirm that they ended in failed installs. Fuel wanted the pubic IP address to have Internet Connectivity, duh. The first attempt was obvious. Re-use the Home Network for Public address. I chose a range of IP addresses that looked safe for the existing natives, turned off DHCP, then ran Validate Network, which checked out OK. Let it rip. All was well until the end. Everything just locked up on the network. Not sure what went wrong. A variation of that configuration should work. I decided not to pursue reuse of the Home Network as the Openstack Public network. My Home Network has users with expectations of using DHCP without Fuel making their laptop an Openstack node.

Yes, for the readers who are paying attention, the Fuel network is different than a combined Public and Home Network. The above is funnier!

An option for alternate configurations could be to use a node for one or more of the NAT translations, opposed to one of the Home NAT routers. However, Home NAT routers are the best option for Ghetto Stack. They’re cheap to buy, cheap to operate, and easy to configure. Other deployments will have large variations in existing network construction, but Ghetto Stack is not about building a production oriented network. At this point, it’s time to buy another NAT router. It’s Midnight on Sunday. Off to Walmart I went, and I found a decent router that I’d be OK with taking with me on my next contract gig. A quick check with Google showed it was $10 cheaper at Best Buy. Say it, and it will happen. Price match, Price match, Price match. I saved one more beer at a pub!

Now we have to actually put some thought into networking. There is no getting around doing a custom network configuration. Fuel needed to be on VLAN 1. That wasn’t going to change. Home Network must have separate DHCP. We shouldn’t have two DHCP servers on one network. Competition in market place is good. DHCP Competition on one network segment is bad. Home Network was assigned to VLAN 2. The NAT router providing Public IP address for Openstack will be double NAT. Google it. Some people on the net have postings that say it’s a bad idea. To them I say, Ghetto stack. The Double NAT router will get its public IP address from Home Network via DHCP on VLAN 2. It will configured to to use the 127.16.0.0/24 range with no DHCP. To hook up the double NAT router, plug one of its private Ethernet ports into the switch on a port configured for VLAN 100.
The Fuel Network, 10.20.0.0/24 will remain on NAT translated by the workstation. The only way this network can be effectively used by an administrator is to have the workstation plugged into all the networks. This fits with a minimal keep-it-simple solution. That is what a minimal fuel network looks like.

An alternative would be to translate an entire class B space. Use a router to make different subnets that can be seen by each other. That is another project for another day.

The end result is the Network Diagram at the beginning of this post. On most recent runs, I removed the second Ethernet connection on the fuel host that is connected to the Home Network. This is the initial boot strap internet connection for the Fuel server before the rest of the network was created. Be sure that Fuel network has public translations. A completely default install of Fuel, and one IP range change for an environment within Fuel will let Fuel fully install Openstack.

This install of Openstack is still not fully functional. That just means Openstack consulting and support will stay in demand for the foreseeable future. At first glance, this may seem really bad. This is no different than the early days of PC. It took many attempts at installing any operating system on a system. Sometimes it took driver downloads or even parts replaced with different brands. This is the normal open systems cost of technical debit shared with whole community. I’m reminded of a buddy who figured out that his permutation of hardware and software worked only if he booted his PC with the CD Rom in the open position, and closed it during the boot process. I never did ask how long he took to figure that out.

Ghetto Stack

posted March 3, 2016