Tuesday, October 11, 2011
111010
I talked with Chris for about 30 minutes about the calculations for baseline load, and then worked the rest of the day on the land use maps and the stats for land use percentage for the meetings they are attending. It took longer than I though to make the maps and to print them out. The printer is very slow, and formatting the tables is done now. I need to figure out where to best put files in the F:/. It is hard because I'm not quite used to the organizational system yet, and don't want to clutter things with GIS files.
Up to Today
110819
Trash Data that Courtney is working on:
F:\SC05\SC05.12 - Trash\Trash Baseline Loading Estimate\Site Selection\Google Earth - Site Locations
This contains the Google Earth files for figuring out which catch basins to monitor
Data for land use:
M:\SFBayRegion\Project MXDs\CB Trash Monitoring.mxd
My working directory for census data:
M:\Alameda\Land Use\Temporary\110819_nick
I need to figure out what to do about the gaps in income data for the census blocks. Could possible use kriging to turn it into a raster, and then create a continuous surface, and then maybe turn it back into a vector shapefile. It would be good if you can have a polygon shapefile and assign it the mean or median value of a continuous surface that is under it, but only for the census blocks that are unknown.
Where John is storing his files from cities:
F:\SC05\SC05.12 - Trash\Trash Baseline Loading Estimate\Baseline Load Calcuation Process\Street Sweeping frequencies- SCVURPPP
Catch basins for San Jose:
M:\Santa Clara\Hydrography\Stormdrain Network\Master\San Jose
ID = 22541 OR
ID = 18557 OR
ID = 27009 OR
ID = 21788 OR
ID = 26376
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110829
Removed from street sweeping:
101
17
237
280
380
680
85
87
880
92
ramp
4WD roads/trails
Connecting Roads
Cemetary Roads
pedestrian paths
driveways
Service Roads
Did not remove:
1
CR 10
109
114
130
152
156
25
35
82
84
9
Cabrillo Hwy
allies
parking lots
Width of Streets
FCC < 40: 50
FCC > 70: 20
Otherwise: 30
Methodology:
Decide which streets to include in the analysis
Columns for street layer:
Width
SS_Radius
buff_dist (Width/2 + SS_Radius)
SS_freq
For each city:
Divide up the city into zones based on SS_freq
Do an intersect based the SS map and the street buffer
Do an intersect on that and LU
Do an intersect on that an income (I need to fill in the gaps)
I was talking to Chris about possibly avoiding having to make the SS maps by just labeling the streets with their frequency. Then while buffering I would dissolve by freqency, and then have a bunch of polygons that overlap. Then I could save each of the polygons as a separate shapefile, and then clip out the parts of the less frequent polygon where they overlap
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110831
To Do:
Work on Baseline/SS analysis
Delineate highways
Figure out what Caltrans sweeps
Emailed Caltrans District 4 a question:
Is it possible to get information on which routes that Caltrans street sweeps in the Bay area, and how often. I am particularly interested in all highways in San Mateo and Santa Clara County including: 1, 109, 114, 130, 152, 156, 25, 35, 82, 84, and 9.
Also, if parts of these highways are swept by cities, then where exactly does the Caltrans sweeping begin and end.
Any information or the contact information of someone who knows would be greatly appreciated,
Thanks,
Nick Zigler
Filled in all streets in Millbrae with a SS_freqency. Now I need to convert the projection to state plane. Then to buffer by the SS_radius, and then erase features of the lower numbers from the higher numbers.
Erase:
1 from 7
1 from 15
7 from 15
1 from 0
7 from 0
15 from 0
Do a union of all
Erase 1 from 7 using the new erase tool
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110902
110901 I was in the field all day with Paul in San Jose. We outlined the drainage areas for 20 catch basin.
Today so far I spent the first hour installing my keyboard drawer and white boards.
My goals for today is to run Millbrae through with the baseline/street sweeping calculation. I also want to delineate some of the catch bains in ArcGIS. This may be easier to to in Google Earth and then transfer it to ArcGIS because the aerials are better.
I also need to read what Laura wrote about FCC classification and try to correlate that with road width
FCC values in the Bay Area:
11, delete
15, delete
16, delete
17, delete
21, delete
23, delete
25, delete
27, delete
30, same as 31
31, Unseparated, generally 4 lane road, or about 50 feet wide
32, same as 31
33, same as 31
35, separated so we need less width, about 30 feet
37, same as 35
38, same as 35
40, same as 41
41, Unseparated, generally 2 lane road, or about 40 feet wide (to be conservative)
43, same as 41
44, same as 41
45, Separated, one lane in each direction, about 25 feet wide (to be conservative)
47, same as 45
50, delete
51, delete
60, ramp or connecting road, delete
61, same as 41
62, same as 41
63, ramp, delete
64, service road, same as 41
66, ferry, delete
69, delete
70, cemetary and other random roads that are mostly dirt, delete
71, walkway, delete
72, walkway, delete
73, alley, ???
74, sevice road, ???
75, parking lots, but only a few, delete?
For next week:
Take this file: C:\GIS\data\transportation\Bay_Area_Streets_RMC_area_SP.shp
Buffer it for the entire bay area, and do an analysis of what land use type get into the buffer, and which do not. Give a spreadsheet to Chris for each of the 9 land use types. Before this, I need to merge the ABAG land uses and then do a re-class for the entire thing so we are all on the same page, and also include some more descriptive labels in the reclass column
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110906
Combined the land uses of Alameda, Contra Costa, Santa Clara, and San Mateo Counties
Did a reclass to include almost everything, and have less polygons labeled as "other"
Cannot dissolve because of memory limitations - may not have to though, it will just keep having 900,000 polygons.
It may work if I convert to raster and then back to polygon
2 hours for making maps of catch basins for when we go out tomorrow
Paul gave me a job to calculate impervious surface area in SM county. Separate between Pacific and Bay Drainages.
"reclass" = 'Commercial' OR "reclass" = 'HDR' OR "reclass" = 'Hvy Ind' OR "reclass" = 'LDR' OR "reclass" = 'Light Ind' OR "reclass" = 'Parks' OR "reclass" = 'Retail' OR "reclass" = 'Roads' OR "reclass" = 'Schools'
Calculated what land uses are in the buffered region, and what is out, using a 30 foot buffer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110908
Worked on landuse for John for a couple hours. Worked on reclassifying for the Bay area as well. All trash stuff
Agriculture
"l_use_code" = 21 OR
"l_use_code" = 22 OR
"l_use_code" = 24 OR
"l_use_code" = 211 OR
"l_use_code" = 212 OR
"l_use_code" = 221 OR
"l_use_code" = 222 OR
"l_use_code" = 223 OR
"l_use_code" = 2111 OR
"l_use_code" = 2112
Barren
"l_use_code" = 72 OR
"l_use_code" = 73 OR
"l_use_code" = 74 OR
"l_use_code" = 75 OR
"l_use_code" = 76 OR
"l_use_code" = 77 OR
"l_use_code" = 751 OR
"l_use_code" = 752 OR
"l_use_code" = 761 OR
"l_use_code" = 762
Colleges and Universities
"l_use_code" = 1232 OR
"l_use_code" = 1233 OR
"l_use_code" = 1234
Commercial
"l_use_code" = 122 OR
"l_use_code" = 124 OR
"l_use_code" = 126 OR
"l_use_code" = 127 OR
"l_use_code" = 128 OR
"l_use_code" = 129 OR
"l_use_code" = 1241 OR
"l_use_code" = 1242 OR
"l_use_code" = 1243 OR
"l_use_code" = 1246 OR
"l_use_code" = 1261 OR
"l_use_code" = 1262 OR
"l_use_code" = 1263 OR
"l_use_code" = 1264 OR
"l_use_code" = 1265 OR
"l_use_code" = 1267 OR
"l_use_code" = 1268 OR
"l_use_code" = 1269 OR
"l_use_code" = 1235
Commercial/Industrial
"l_use_code" = 15
Comunication/Power Facilities
"l_use_code" = 148 OR
"l_use_code" = 1451 OR
"l_use_code" = 1452 OR
"l_use_code" = 1453 OR
"l_use_code" = 1483
Forest
"l_use_code" = 41 OR
"l_use_code" = 42 OR
"l_use_code" = 43 OR
"l_use_code" = 411 OR
"l_use_code" = 421 OR
"l_use_code" = 431
General Residential
"l_use_code" = 1751
Heavy Industrial
"l_use_code" = 131
High Density Residential
"l_use_code" = 11 OR
"l_use_code" = 113 OR
"l_use_code" = 114 OR
"l_use_code" = 115 OR
"l_use_code" = 118 OR
"l_use_code" = 119
K-12 Schools
"l_use_code" = 123 OR
"l_use_code" = 1231
Light Industrial
"l_use_code" = 13 OR
"l_use_code" = 132 OR
"l_use_code" = 134 OR
"l_use_code" = 135
Low Density Residential
"l_use_code" = 111 OR
"l_use_code" = 112
Military
"l_use_code" = 125 OR
"l_use_code" = 1253 OR
"l_use_code" = 1254 OR
"l_use_code" = 1257 OR
"l_use_code" = 1258 OR
"l_use_code" = 1259
Other
"l_use_code" = 0 OR
"l_use_code" = 12
Rangeland
"l_use_code" = 31 OR
"l_use_code" = 32 OR
"l_use_code" = 33 OR
"l_use_code" = 311 OR
"l_use_code" = 321 OR
"l_use_code" = 331
Residential/Commercial
"l_use_code" = 16 OR
"l_use_code" = 161 OR
"l_use_code" = 162
Retail
"l_use_code" = 121
State Facilities
"l_use_code" = 1247 OR
"l_use_code" = 1248 OR
"l_use_code" = 1249
Transportation - Airports
"l_use_code" = 143 OR
"l_use_code" = 1431 OR
"l_use_code" = 1432 OR
"l_use_code" = 1433 OR
"l_use_code" = 1434 OR
"l_use_code" = 1435 OR
"l_use_code" = 1436 OR
"l_use_code" = 1437 OR
"l_use_code" = 1438
Transportation - Other
"l_use_code" = 14 OR
"l_use_code" = 141
Transportation - Parking Lots
"l_use_code" = 1413 OR
"l_use_code" = 1414 OR
"l_use_code" = 1415 OR
"l_use_code" = 1416
Transportation - Ports
"l_use_code" = 144 OR
"l_use_code" = 1441 OR
"l_use_code" = 1442 OR
"l_use_code" = 1443 OR
"l_use_code" = 1444 OR
"l_use_code" = 1445 OR
"l_use_code" = 1448
Transportation - Rail
"l_use_code" = 142 OR
"l_use_code" = 1421 OR
"l_use_code" = 1422
Transportation - Roads
"l_use_code" = 1411 OR
"l_use_code" = 1418
Urban Open
"l_use_code" = 17 OR
"l_use_code" = 171 OR
"l_use_code" = 172 OR
"l_use_code" = 174 OR
"l_use_code" = 175 OR
"l_use_code" = 1711 OR
"l_use_code" = 1712 OR
"l_use_code" = 1713 OR
"l_use_code" = 1752 OR
"l_use_code" = 1753
Urban Parks
"l_use_code" = 173
Wastewater Facililties
"l_use_code" = 1461 OR
"l_use_code" = 1462 OR
"l_use_code" = 1463
Water
"l_use_code" = 5 OR
"l_use_code" = 51 OR
"l_use_code" = 52 OR
"l_use_code" = 53 OR
"l_use_code" = 54 OR
"l_use_code" = 55 OR
"l_use_code" = 56 OR
"l_use_code" = 561 OR
"l_use_code" = 563
Water Supply Faciliites
"l_use_code" = 1471 OR
"l_use_code" = 1473 OR
"l_use_code" = 1474 OR
"l_use_code" = 147
Wetland
"l_use_code" = 61 OR
"l_use_code" = 62 OR
"l_use_code" = 63 OR
"l_use_code" = 64
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110912
Need to work on street sweeping areas for cities
Created a city boundary for all the cities in Santa Clara County
Los Altos
"GEONAME_L" = 'Los Altos' OR "GEONAME_R" = 'Los Altos'
I'm not sure what they mean my Civic Center and Downtown Plazas, so I can't figure out what is swept once a week. It is all set to once a month for now in these areas.
There is no indication about parking enforcement
Los Altos Hills
"GEONAME_L" = 'Los Altos Hills' OR "GEONAME_R" = 'Los Altos Hills'
Cupertino
"GEONAME_L" = 'Cupertino' OR "GEONAME_R" = 'Cupertino'
Not sure about the roads on the border: Homestead Rd, Bollinger Rd, Sunnyvale-Saratoga Rd, Prospect Rd - Do they sweep only one side of the road?
Easy to tell what is enforced and not enforced
Milpitas
Good map that says exactly what is swept and when
"GEONAME_L" = 'Milpitas' OR "GEONAME_R" = 'Milpitas'
Just to confirm, the city sweeps E Calaveras Blvd/Hwy 237 from I-880 going east
There is no information about parking enforcement. The website infers that there is no enforcement.
Mountain View
"GEONAME_L" = 'Mountain View' OR "GEONAME_R" = 'Mountain View' AND "SS_Freq" = -999
Not a very detailed map at all. It is difficult to tell exactly what streets are swept.
The website infers there is no enforcement, although I know there is since I lived there.
Palo Alto
"GEONAME_L" = 'Palo Alto' OR "GEONAME_R" = 'Palo Alto'
Decent map of which streets are what frequency.
There is an enforcement map, but I'm not sure if it is correct because it is about 6 years old, and it hasn't been updated. There is also an additional rule where temporary no parking signs are put up every 2-3 months for SS. This is only for a handful of streets.
West Valley
Saratoga
"GEONAME_L" = 'Saratoga' OR "GEONAME_R" = 'Saratoga'
It appears they also sweep a section of Hwy 9, which is the road that goes through downtown Saratoga
Cambell
"GEONAME_L" = 'Cambell' OR "GEONAME_R" = 'Cambell'
Paper map on 5 sheets. It is tough to tell exactly what streets are swept since it was scanned in black and white and the original is in color, plus it is split into 5 sheets. What are the exact street downtown that are swept twice a month. I can't tell from the map. What do the dotted lines mean on the map?
Los Gatos
"GEONAME_L" = 'Los Gatos' OR "GEONAME_R" = 'Los Gatos'
The map doesn't indicate private streets that are not swept. It also doesn't indicate the exact streets downtown that are swept weekly. I guessed on this and made the main street once a week, but we need more data
San Jose
"GEONAME_L" = 'San Jose' OR "GEONAME_R" = 'San Jose'
Sunnyvale
"GEONAME_L" = 'Sunnyvale' OR "GEONAME_R" = 'Sunnyvale'
Foster City
"GEONAME_L" = 'Foster City' OR "GEONAME_R" = 'Foster City'
Daly City
"GEONAME_L" = 'Daly City' OR "GEONAME_R" = 'Daly City'
San Bruno
("GEONAME_L" = 'San Bruno' OR "GEONAME_R" = 'San Bruno') AND
("GEONAME_L" = 'San Jose' OR "GEONAME_R" = 'San Jose') AND "SS_Agency" = ''
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110913
Expressways in SC County:
Almaden, Capitol, Central, Foothill, Hegenberger, Lawrence, Montague, Oregon, San Tomas, Southwest,
In San Mateo: Woodside
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110916
110915 I was in San Jose in the field outlining catch basins all day
Created a shapefile of all San Jose catch basin areas at F:\SC05\SC05.12 - Trash\Trash Baseline Loading Estimate\Site Selection\Catch Basin Areas\
I've been editing the one that ends in SP to fully attribute the data.
Also saved and printed out maps of catch basins in San Mateo and Brisbane so we can go out on Monday
We also need maps made of the ones in San Leandro, 25 I guess
I still need to do the street sweeping for sunnyvale, and update Mountain View. It's probably best to wait until we get more data in, so I can spend an entire day doing it.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110920
Worked on street sweeping all day
Finished sunnyvale, Los Altos Hills, Millbrae, San Carlos, and half done with Daly City - right now all the streets that are on the list are set to once a week, and this needs to change for some of the streets
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110921
Worked on Street sweeping for Daly City and San Bruno for about 6 hours. Worked on catch basin areas for 2 hours
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110922
Worked on catch basin delineation most of the day.
I'm also trying to organize my work.
Number = 0 ' Initialize variable.
If [FCC] = "A40" Then
TEMP = 1
Else
TEMP = 3
End If
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110923
Worked in office all day
Trying to figure out baseline analysis
Looked at shapefile for businesses
Trying to get rid of transportation layer
CON(isnull([]), FOCALMEAN([], rectangle,4,4), [])
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110926
Con(IsNull("small_raster2"),FocalStatistics("small_raster2",NbrCircle(5),"MAJORITY"),"small_raster2")
Con(IsNull("no_gaps4"),FocalStatistics("no_gaps4",NbrCircle(3),"MAJORITY"),"no_gaps4")
Con(IsNull("no_gaps5"),FocalStatistics("no_gaps5",NbrCircle(3),"MAJORITY"),"no_gaps5")
experiment number 2 - larger area:
convert to raster ->SMC_raster
Con(IsNull("SMC_raster"),FocalStatistics("SMC_raster",NbrCircle(4),"MAJORITY"),"SMC_raster")
Con(IsNull("SMC_raster_a4"),FocalStatistics("SMC_raster_a4",NbrCircle(6),"MAJORITY"),"SMC_raster_a4")
experiment number 3 - larger area:
Con(IsNull("LU_raster"),FocalStatistics("LU_raster",NbrCircle(4),"MAJORITY"),"LU_raster")
lu_raster_a4
Con(IsNull("LU_raster_a4"),FocalStatistics("LU_raster_a4",NbrCircle(4),"MAJORITY"),"LU_raster_a4")
lu_raster_b5
Four County Land Use to raster, use 16.4 as size
Have reclass as the value
Fill in the void areas by running the gap filling algorithm with several different circle radius'
lu_raster2 = setNull("lu_raster1" == 26, "lu_raster1")
lu_raster3 = Con(IsNull("lu_raster2"), FocalStatistics("lu_raster2", NbrCircle(4), "MAJORITY"), "lu_raster2")
lu_raster5 = Con(IsNull("raster4"), FocalStatistics("raster4", NbrCircle(5), "MAJORITY"), "raster4")
lu_raster6 = Con(IsNull("lu_raster5"), FocalStatistics("lu_raster5", NbrCircle(7), "MAJORITY"), "lu_raster5")
lu_raster7 = Con(IsNull("lu_raster6"), FocalStatistics("lu_raster6", NbrCircle(8), "MAJORITY"), "lu_raster6")
lu_raster8 = Con(IsNull("lu_raster7"), FocalStatistics("lu_raster7", NbrRectangle(4), "MAJORITY"), "lu_raster7")
Run and algorithm to get rid of islands to clean stuff up.
Con(if "raster" != FocalStatistics("raster",NbrCircle(4),"MAJORITY")
Convert the raster back into a shapefile, using the simplify polygons algorithm
Run an intersect
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110927
VALUE COUNT RECLASS
1 3360559 Other
2 33343716 High Density Residential
3 4710861 Light Industrial
4 600817 Transportation - Other
5 532339 Commercial/Industrial
6 77990 Residential/Commercial
7 8822903 Urban Open
8 17330136 Agriculture
9 114186732 Rangeland
10 75120551 Forest
11 7004923 Water
12 7391940 Wetland
13 2284760 Barren
14 17543944 Low Density Residential
15 3104691 Retail
16 4551080 Commercial
17 2766018 K-12 Schools
18 2358253 Military
19 2516488 Heavy Industrial
20 184462 Transportation - Rail
21 1428715 Transportation - Airports
22 155629 Comunication/Power Facilities
23 4925551 Urban Parks
24 1276534 Colleges and Universities
25 28779 State Facilities
26 19784903 Transportation - Roads
27 214973 Transportation - Parking Lots
28 308700 Transportation - Ports
29 104699 Wastewater Facililties
30 39076 Water Supply Faciliites
31 1640249 General Residential
Working on Land Use file, and converting it from a raster to a shapefile. I have a simplified version and a non-simplified version
Trying to get census data and land use data joined together
Working on a Sample City using San Bruno, in the Data/Sample_City folder. I created a loading factor spreadsheet, so I can assign each landuse a loading in GIS
This folder also contains the Land Use reclass table
Working on the methodology for calculating trash loads and SS reductions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110928
Field calculator to get the street sweeping factor attribute
SS_factor(!SS_freq!, !enforcemnt!)
def SS_factor(freqency, enforcement):
enforcement = enforcement.strip()
enforcement = enforcement.lower()
if frequency == 0 or frequency == -999:
return 1.0
elif enforcement == "yes":
return 0.06 * math.ln(frequency) + 0.5
else:
return 0.11 * math.ln(frequency) + 0.1
I created a model for the process, although there is a couple major issues. First, there is no good way to get rid of everlaps in the buffers. The best workaround that I can figure is to perform a Union with itself, sort by one of the Geometry fields, and then delete the overlaps that are not needed. This creates a file with invalid geometry, that cannot be repaired. The data can be exported, and this seems to work. This file can then be used as the input to the Model. I still need to verify my above script for the field calculator. I also need a process to add the regions not in the buffer back in. This should go in Model1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
111004
Monday I was in the field in San Leandro all day
Tuesday I was in the field in Berkeley, Oakland, Fremont, and Dublin collection catch basin info by myself
Today is an office day organizing all of the catch basin info and trying to get it to courtney
I still need to edit the land use data to get a more accurate land use for the catch basins
The analysis is being done at:
C:\GIS\data\Catch_Basins\shapefile\CBs8
There will be four sets of data:
a. Original LU with buffer
b. Original LU with CB area
c. Edited LU with buffer
d. Edited LU with CB area
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
111005
Working in the office all day on catch basin information and processing landuse
Also worked on street sweeping freqency for the spreadsheet for a few hours. John needs to call the cities and get the freqency for some of the towns
Trash Data that Courtney is working on:
F:\SC05\SC05.12 - Trash\Trash Baseline Loading Estimate\Site Selection\Google Earth - Site Locations
This contains the Google Earth files for figuring out which catch basins to monitor
Data for land use:
M:\SFBayRegion\Project MXDs\CB Trash Monitoring.mxd
My working directory for census data:
M:\Alameda\Land Use\Temporary\110819_nick
I need to figure out what to do about the gaps in income data for the census blocks. Could possible use kriging to turn it into a raster, and then create a continuous surface, and then maybe turn it back into a vector shapefile. It would be good if you can have a polygon shapefile and assign it the mean or median value of a continuous surface that is under it, but only for the census blocks that are unknown.
Where John is storing his files from cities:
F:\SC05\SC05.12 - Trash\Trash Baseline Loading Estimate\Baseline Load Calcuation Process\Street Sweeping frequencies- SCVURPPP
Catch basins for San Jose:
M:\Santa Clara\Hydrography\Stormdrain Network\Master\San Jose
ID = 22541 OR
ID = 18557 OR
ID = 27009 OR
ID = 21788 OR
ID = 26376
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110829
Removed from street sweeping:
101
17
237
280
380
680
85
87
880
92
ramp
4WD roads/trails
Connecting Roads
Cemetary Roads
pedestrian paths
driveways
Service Roads
Did not remove:
1
CR 10
109
114
130
152
156
25
35
82
84
9
Cabrillo Hwy
allies
parking lots
Width of Streets
FCC < 40: 50
FCC > 70: 20
Otherwise: 30
Methodology:
Decide which streets to include in the analysis
Columns for street layer:
Width
SS_Radius
buff_dist (Width/2 + SS_Radius)
SS_freq
For each city:
Divide up the city into zones based on SS_freq
Do an intersect based the SS map and the street buffer
Do an intersect on that and LU
Do an intersect on that an income (I need to fill in the gaps)
I was talking to Chris about possibly avoiding having to make the SS maps by just labeling the streets with their frequency. Then while buffering I would dissolve by freqency, and then have a bunch of polygons that overlap. Then I could save each of the polygons as a separate shapefile, and then clip out the parts of the less frequent polygon where they overlap
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110831
To Do:
Work on Baseline/SS analysis
Delineate highways
Figure out what Caltrans sweeps
Emailed Caltrans District 4 a question:
Is it possible to get information on which routes that Caltrans street sweeps in the Bay area, and how often. I am particularly interested in all highways in San Mateo and Santa Clara County including: 1, 109, 114, 130, 152, 156, 25, 35, 82, 84, and 9.
Also, if parts of these highways are swept by cities, then where exactly does the Caltrans sweeping begin and end.
Any information or the contact information of someone who knows would be greatly appreciated,
Thanks,
Nick Zigler
Filled in all streets in Millbrae with a SS_freqency. Now I need to convert the projection to state plane. Then to buffer by the SS_radius, and then erase features of the lower numbers from the higher numbers.
Erase:
1 from 7
1 from 15
7 from 15
1 from 0
7 from 0
15 from 0
Do a union of all
Erase 1 from 7 using the new erase tool
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110902
110901 I was in the field all day with Paul in San Jose. We outlined the drainage areas for 20 catch basin.
Today so far I spent the first hour installing my keyboard drawer and white boards.
My goals for today is to run Millbrae through with the baseline/street sweeping calculation. I also want to delineate some of the catch bains in ArcGIS. This may be easier to to in Google Earth and then transfer it to ArcGIS because the aerials are better.
I also need to read what Laura wrote about FCC classification and try to correlate that with road width
FCC values in the Bay Area:
11, delete
15, delete
16, delete
17, delete
21, delete
23, delete
25, delete
27, delete
30, same as 31
31, Unseparated, generally 4 lane road, or about 50 feet wide
32, same as 31
33, same as 31
35, separated so we need less width, about 30 feet
37, same as 35
38, same as 35
40, same as 41
41, Unseparated, generally 2 lane road, or about 40 feet wide (to be conservative)
43, same as 41
44, same as 41
45, Separated, one lane in each direction, about 25 feet wide (to be conservative)
47, same as 45
50, delete
51, delete
60, ramp or connecting road, delete
61, same as 41
62, same as 41
63, ramp, delete
64, service road, same as 41
66, ferry, delete
69, delete
70, cemetary and other random roads that are mostly dirt, delete
71, walkway, delete
72, walkway, delete
73, alley, ???
74, sevice road, ???
75, parking lots, but only a few, delete?
For next week:
Take this file: C:\GIS\data\transportation\Bay_Area_Streets_RMC_area_SP.shp
Buffer it for the entire bay area, and do an analysis of what land use type get into the buffer, and which do not. Give a spreadsheet to Chris for each of the 9 land use types. Before this, I need to merge the ABAG land uses and then do a re-class for the entire thing so we are all on the same page, and also include some more descriptive labels in the reclass column
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110906
Combined the land uses of Alameda, Contra Costa, Santa Clara, and San Mateo Counties
Did a reclass to include almost everything, and have less polygons labeled as "other"
Cannot dissolve because of memory limitations - may not have to though, it will just keep having 900,000 polygons.
It may work if I convert to raster and then back to polygon
2 hours for making maps of catch basins for when we go out tomorrow
Paul gave me a job to calculate impervious surface area in SM county. Separate between Pacific and Bay Drainages.
"reclass" = 'Commercial' OR "reclass" = 'HDR' OR "reclass" = 'Hvy Ind' OR "reclass" = 'LDR' OR "reclass" = 'Light Ind' OR "reclass" = 'Parks' OR "reclass" = 'Retail' OR "reclass" = 'Roads' OR "reclass" = 'Schools'
Calculated what land uses are in the buffered region, and what is out, using a 30 foot buffer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110908
Worked on landuse for John for a couple hours. Worked on reclassifying for the Bay area as well. All trash stuff
Agriculture
"l_use_code" = 21 OR
"l_use_code" = 22 OR
"l_use_code" = 24 OR
"l_use_code" = 211 OR
"l_use_code" = 212 OR
"l_use_code" = 221 OR
"l_use_code" = 222 OR
"l_use_code" = 223 OR
"l_use_code" = 2111 OR
"l_use_code" = 2112
Barren
"l_use_code" = 72 OR
"l_use_code" = 73 OR
"l_use_code" = 74 OR
"l_use_code" = 75 OR
"l_use_code" = 76 OR
"l_use_code" = 77 OR
"l_use_code" = 751 OR
"l_use_code" = 752 OR
"l_use_code" = 761 OR
"l_use_code" = 762
Colleges and Universities
"l_use_code" = 1232 OR
"l_use_code" = 1233 OR
"l_use_code" = 1234
Commercial
"l_use_code" = 122 OR
"l_use_code" = 124 OR
"l_use_code" = 126 OR
"l_use_code" = 127 OR
"l_use_code" = 128 OR
"l_use_code" = 129 OR
"l_use_code" = 1241 OR
"l_use_code" = 1242 OR
"l_use_code" = 1243 OR
"l_use_code" = 1246 OR
"l_use_code" = 1261 OR
"l_use_code" = 1262 OR
"l_use_code" = 1263 OR
"l_use_code" = 1264 OR
"l_use_code" = 1265 OR
"l_use_code" = 1267 OR
"l_use_code" = 1268 OR
"l_use_code" = 1269 OR
"l_use_code" = 1235
Commercial/Industrial
"l_use_code" = 15
Comunication/Power Facilities
"l_use_code" = 148 OR
"l_use_code" = 1451 OR
"l_use_code" = 1452 OR
"l_use_code" = 1453 OR
"l_use_code" = 1483
Forest
"l_use_code" = 41 OR
"l_use_code" = 42 OR
"l_use_code" = 43 OR
"l_use_code" = 411 OR
"l_use_code" = 421 OR
"l_use_code" = 431
General Residential
"l_use_code" = 1751
Heavy Industrial
"l_use_code" = 131
High Density Residential
"l_use_code" = 11 OR
"l_use_code" = 113 OR
"l_use_code" = 114 OR
"l_use_code" = 115 OR
"l_use_code" = 118 OR
"l_use_code" = 119
K-12 Schools
"l_use_code" = 123 OR
"l_use_code" = 1231
Light Industrial
"l_use_code" = 13 OR
"l_use_code" = 132 OR
"l_use_code" = 134 OR
"l_use_code" = 135
Low Density Residential
"l_use_code" = 111 OR
"l_use_code" = 112
Military
"l_use_code" = 125 OR
"l_use_code" = 1253 OR
"l_use_code" = 1254 OR
"l_use_code" = 1257 OR
"l_use_code" = 1258 OR
"l_use_code" = 1259
Other
"l_use_code" = 0 OR
"l_use_code" = 12
Rangeland
"l_use_code" = 31 OR
"l_use_code" = 32 OR
"l_use_code" = 33 OR
"l_use_code" = 311 OR
"l_use_code" = 321 OR
"l_use_code" = 331
Residential/Commercial
"l_use_code" = 16 OR
"l_use_code" = 161 OR
"l_use_code" = 162
Retail
"l_use_code" = 121
State Facilities
"l_use_code" = 1247 OR
"l_use_code" = 1248 OR
"l_use_code" = 1249
Transportation - Airports
"l_use_code" = 143 OR
"l_use_code" = 1431 OR
"l_use_code" = 1432 OR
"l_use_code" = 1433 OR
"l_use_code" = 1434 OR
"l_use_code" = 1435 OR
"l_use_code" = 1436 OR
"l_use_code" = 1437 OR
"l_use_code" = 1438
Transportation - Other
"l_use_code" = 14 OR
"l_use_code" = 141
Transportation - Parking Lots
"l_use_code" = 1413 OR
"l_use_code" = 1414 OR
"l_use_code" = 1415 OR
"l_use_code" = 1416
Transportation - Ports
"l_use_code" = 144 OR
"l_use_code" = 1441 OR
"l_use_code" = 1442 OR
"l_use_code" = 1443 OR
"l_use_code" = 1444 OR
"l_use_code" = 1445 OR
"l_use_code" = 1448
Transportation - Rail
"l_use_code" = 142 OR
"l_use_code" = 1421 OR
"l_use_code" = 1422
Transportation - Roads
"l_use_code" = 1411 OR
"l_use_code" = 1418
Urban Open
"l_use_code" = 17 OR
"l_use_code" = 171 OR
"l_use_code" = 172 OR
"l_use_code" = 174 OR
"l_use_code" = 175 OR
"l_use_code" = 1711 OR
"l_use_code" = 1712 OR
"l_use_code" = 1713 OR
"l_use_code" = 1752 OR
"l_use_code" = 1753
Urban Parks
"l_use_code" = 173
Wastewater Facililties
"l_use_code" = 1461 OR
"l_use_code" = 1462 OR
"l_use_code" = 1463
Water
"l_use_code" = 5 OR
"l_use_code" = 51 OR
"l_use_code" = 52 OR
"l_use_code" = 53 OR
"l_use_code" = 54 OR
"l_use_code" = 55 OR
"l_use_code" = 56 OR
"l_use_code" = 561 OR
"l_use_code" = 563
Water Supply Faciliites
"l_use_code" = 1471 OR
"l_use_code" = 1473 OR
"l_use_code" = 1474 OR
"l_use_code" = 147
Wetland
"l_use_code" = 61 OR
"l_use_code" = 62 OR
"l_use_code" = 63 OR
"l_use_code" = 64
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110912
Need to work on street sweeping areas for cities
Created a city boundary for all the cities in Santa Clara County
Los Altos
"GEONAME_L" = 'Los Altos' OR "GEONAME_R" = 'Los Altos'
I'm not sure what they mean my Civic Center and Downtown Plazas, so I can't figure out what is swept once a week. It is all set to once a month for now in these areas.
There is no indication about parking enforcement
Los Altos Hills
"GEONAME_L" = 'Los Altos Hills' OR "GEONAME_R" = 'Los Altos Hills'
Cupertino
"GEONAME_L" = 'Cupertino' OR "GEONAME_R" = 'Cupertino'
Not sure about the roads on the border: Homestead Rd, Bollinger Rd, Sunnyvale-Saratoga Rd, Prospect Rd - Do they sweep only one side of the road?
Easy to tell what is enforced and not enforced
Milpitas
Good map that says exactly what is swept and when
"GEONAME_L" = 'Milpitas' OR "GEONAME_R" = 'Milpitas'
Just to confirm, the city sweeps E Calaveras Blvd/Hwy 237 from I-880 going east
There is no information about parking enforcement. The website infers that there is no enforcement.
Mountain View
"GEONAME_L" = 'Mountain View' OR "GEONAME_R" = 'Mountain View' AND "SS_Freq" = -999
Not a very detailed map at all. It is difficult to tell exactly what streets are swept.
The website infers there is no enforcement, although I know there is since I lived there.
Palo Alto
"GEONAME_L" = 'Palo Alto' OR "GEONAME_R" = 'Palo Alto'
Decent map of which streets are what frequency.
There is an enforcement map, but I'm not sure if it is correct because it is about 6 years old, and it hasn't been updated. There is also an additional rule where temporary no parking signs are put up every 2-3 months for SS. This is only for a handful of streets.
West Valley
Saratoga
"GEONAME_L" = 'Saratoga' OR "GEONAME_R" = 'Saratoga'
It appears they also sweep a section of Hwy 9, which is the road that goes through downtown Saratoga
Cambell
"GEONAME_L" = 'Cambell' OR "GEONAME_R" = 'Cambell'
Paper map on 5 sheets. It is tough to tell exactly what streets are swept since it was scanned in black and white and the original is in color, plus it is split into 5 sheets. What are the exact street downtown that are swept twice a month. I can't tell from the map. What do the dotted lines mean on the map?
Los Gatos
"GEONAME_L" = 'Los Gatos' OR "GEONAME_R" = 'Los Gatos'
The map doesn't indicate private streets that are not swept. It also doesn't indicate the exact streets downtown that are swept weekly. I guessed on this and made the main street once a week, but we need more data
San Jose
"GEONAME_L" = 'San Jose' OR "GEONAME_R" = 'San Jose'
Sunnyvale
"GEONAME_L" = 'Sunnyvale' OR "GEONAME_R" = 'Sunnyvale'
Foster City
"GEONAME_L" = 'Foster City' OR "GEONAME_R" = 'Foster City'
Daly City
"GEONAME_L" = 'Daly City' OR "GEONAME_R" = 'Daly City'
San Bruno
("GEONAME_L" = 'San Bruno' OR "GEONAME_R" = 'San Bruno') AND
("GEONAME_L" = 'San Jose' OR "GEONAME_R" = 'San Jose') AND "SS_Agency" = ''
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110913
Expressways in SC County:
Almaden, Capitol, Central, Foothill, Hegenberger, Lawrence, Montague, Oregon, San Tomas, Southwest,
In San Mateo: Woodside
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110916
110915 I was in San Jose in the field outlining catch basins all day
Created a shapefile of all San Jose catch basin areas at F:\SC05\SC05.12 - Trash\Trash Baseline Loading Estimate\Site Selection\Catch Basin Areas\
I've been editing the one that ends in SP to fully attribute the data.
Also saved and printed out maps of catch basins in San Mateo and Brisbane so we can go out on Monday
We also need maps made of the ones in San Leandro, 25 I guess
I still need to do the street sweeping for sunnyvale, and update Mountain View. It's probably best to wait until we get more data in, so I can spend an entire day doing it.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110920
Worked on street sweeping all day
Finished sunnyvale, Los Altos Hills, Millbrae, San Carlos, and half done with Daly City - right now all the streets that are on the list are set to once a week, and this needs to change for some of the streets
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110921
Worked on Street sweeping for Daly City and San Bruno for about 6 hours. Worked on catch basin areas for 2 hours
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110922
Worked on catch basin delineation most of the day.
I'm also trying to organize my work.
Number = 0 ' Initialize variable.
If [FCC] = "A40" Then
TEMP = 1
Else
TEMP = 3
End If
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110923
Worked in office all day
Trying to figure out baseline analysis
Looked at shapefile for businesses
Trying to get rid of transportation layer
CON(isnull([
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110926
Con(IsNull("small_raster2"),FocalStatistics("small_raster2",NbrCircle(5),"MAJORITY"),"small_raster2")
Con(IsNull("no_gaps4"),FocalStatistics("no_gaps4",NbrCircle(3),"MAJORITY"),"no_gaps4")
Con(IsNull("no_gaps5"),FocalStatistics("no_gaps5",NbrCircle(3),"MAJORITY"),"no_gaps5")
experiment number 2 - larger area:
convert to raster ->SMC_raster
Con(IsNull("SMC_raster"),FocalStatistics("SMC_raster",NbrCircle(4),"MAJORITY"),"SMC_raster")
Con(IsNull("SMC_raster_a4"),FocalStatistics("SMC_raster_a4",NbrCircle(6),"MAJORITY"),"SMC_raster_a4")
experiment number 3 - larger area:
Con(IsNull("LU_raster"),FocalStatistics("LU_raster",NbrCircle(4),"MAJORITY"),"LU_raster")
lu_raster_a4
Con(IsNull("LU_raster_a4"),FocalStatistics("LU_raster_a4",NbrCircle(4),"MAJORITY"),"LU_raster_a4")
lu_raster_b5
Four County Land Use to raster, use 16.4 as size
Have reclass as the value
Fill in the void areas by running the gap filling algorithm with several different circle radius'
lu_raster2 = setNull("lu_raster1" == 26, "lu_raster1")
lu_raster3 = Con(IsNull("lu_raster2"), FocalStatistics("lu_raster2", NbrCircle(4), "MAJORITY"), "lu_raster2")
lu_raster5 = Con(IsNull("raster4"), FocalStatistics("raster4", NbrCircle(5), "MAJORITY"), "raster4")
lu_raster6 = Con(IsNull("lu_raster5"), FocalStatistics("lu_raster5", NbrCircle(7), "MAJORITY"), "lu_raster5")
lu_raster7 = Con(IsNull("lu_raster6"), FocalStatistics("lu_raster6", NbrCircle(8), "MAJORITY"), "lu_raster6")
lu_raster8 = Con(IsNull("lu_raster7"), FocalStatistics("lu_raster7", NbrRectangle(4), "MAJORITY"), "lu_raster7")
Run and algorithm to get rid of islands to clean stuff up.
Con(if "raster" != FocalStatistics("raster",NbrCircle(4),"MAJORITY")
Convert the raster back into a shapefile, using the simplify polygons algorithm
Run an intersect
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110927
VALUE COUNT RECLASS
1 3360559 Other
2 33343716 High Density Residential
3 4710861 Light Industrial
4 600817 Transportation - Other
5 532339 Commercial/Industrial
6 77990 Residential/Commercial
7 8822903 Urban Open
8 17330136 Agriculture
9 114186732 Rangeland
10 75120551 Forest
11 7004923 Water
12 7391940 Wetland
13 2284760 Barren
14 17543944 Low Density Residential
15 3104691 Retail
16 4551080 Commercial
17 2766018 K-12 Schools
18 2358253 Military
19 2516488 Heavy Industrial
20 184462 Transportation - Rail
21 1428715 Transportation - Airports
22 155629 Comunication/Power Facilities
23 4925551 Urban Parks
24 1276534 Colleges and Universities
25 28779 State Facilities
26 19784903 Transportation - Roads
27 214973 Transportation - Parking Lots
28 308700 Transportation - Ports
29 104699 Wastewater Facililties
30 39076 Water Supply Faciliites
31 1640249 General Residential
Working on Land Use file, and converting it from a raster to a shapefile. I have a simplified version and a non-simplified version
Trying to get census data and land use data joined together
Working on a Sample City using San Bruno, in the Data/Sample_City folder. I created a loading factor spreadsheet, so I can assign each landuse a loading in GIS
This folder also contains the Land Use reclass table
Working on the methodology for calculating trash loads and SS reductions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110928
Field calculator to get the street sweeping factor attribute
SS_factor(!SS_freq!, !enforcemnt!)
def SS_factor(freqency, enforcement):
enforcement = enforcement.strip()
enforcement = enforcement.lower()
if frequency == 0 or frequency == -999:
return 1.0
elif enforcement == "yes":
return 0.06 * math.ln(frequency) + 0.5
else:
return 0.11 * math.ln(frequency) + 0.1
I created a model for the process, although there is a couple major issues. First, there is no good way to get rid of everlaps in the buffers. The best workaround that I can figure is to perform a Union with itself, sort by one of the Geometry fields, and then delete the overlaps that are not needed. This creates a file with invalid geometry, that cannot be repaired. The data can be exported, and this seems to work. This file can then be used as the input to the Model. I still need to verify my above script for the field calculator. I also need a process to add the regions not in the buffer back in. This should go in Model1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
111004
Monday I was in the field in San Leandro all day
Tuesday I was in the field in Berkeley, Oakland, Fremont, and Dublin collection catch basin info by myself
Today is an office day organizing all of the catch basin info and trying to get it to courtney
I still need to edit the land use data to get a more accurate land use for the catch basins
The analysis is being done at:
C:\GIS\data\Catch_Basins\shapefile\CBs8
There will be four sets of data:
a. Original LU with buffer
b. Original LU with CB area
c. Edited LU with buffer
d. Edited LU with CB area
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
111005
Working in the office all day on catch basin information and processing landuse
Also worked on street sweeping freqency for the spreadsheet for a few hours. John needs to call the cities and get the freqency for some of the towns
Subscribe to:
Posts (Atom)