阳光沙滩-让学习编程变得更加简单

全部文章问答分享共找到416个相关内容

实现顺序播放音频，开始播放了之后，界面就卡住了（久了之后就会提示：应用没有响应，是否关闭），只有音频在不停地播放

2021-06-05 15:44
                ·
                音频 / MediaPlayer

[文章] ElasticSearch安装以及相关操作

[TOC]一.ElasticSearch介绍1.1引言在海量数据中执行搜索功能时，如果使用MySQL，效率太低。如果关键字输入的不准确，一样可以搜索到想要的数据。将关键字，以红色的样式展示。1.2ES的介绍ES是基于Java语言并且基于Lucene编写的搜索引擎框架，他提供了分布式的全文搜索功能，提供了一个统一的基于RESTful风格的WEB接口，官方客户端也对多种语言都提供了相应的API。Lucene：是一个搜索引擎的底层。全文搜索：将一段词语进行分词，并且将分出的单个词语统一的放到一个分词库中，在搜索时，根据关键字去分词库中检索，找到匹配的内容。二.ElasticSearch安装2.1安装ES&Kibana图形化界面version:"3.1"services:elasticsearch:image:daocloud.io/library/elasticsearch:6.5.4restart:alwayscontainer_name:elasticsearchports:-9200:9200kibana:image:daocloud.io/library/kibana:6.5.4restart:alwayscontainer_name:kibanaports:-5601:5601environment:-elasticsearch_url=http://192.168.0.17:9200depends_on:-elasticsearch2.2安装IK分词器下载地址：https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.4/elasticsearch-analysis-ik-6.5.4.zip进入ES容器内部，并跳转到bin目录下，执行bin目录下的脚本文件：./elasticsearch-plugininstallhttps://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.4/elasticsearch-analysis-ik-6.5.4.zip安装IK之后一定要重启ES。三.ES的基本操作3.1ES的结构３.１.１索引Index，分片和备份ES的服务中，可以创建多个索引。每个索引默认会被分成5片存储。每一个分片都会存在至少一个备份分片。备份分片默认不会帮助检索数据，当ES检索压力特别大的时候，备份分片才会帮助检索数据。备份分片必须放在不同的服务器中。3.1.2类型Type一个索引下，可以创建多个类型。Ps：根据版本不同，类型的创建也不同3.1.3文档Doc一个类型下，可以有多个文档。这个文档就类似于MySQL表中的多行数据。3.1.4属性field一个文档下，可以有多个属性。这个文档就类似于MySQL表中的一行数据存在多个列。3.2操作ES的RESTful语法GET请求：http://ip:port/index：查询索引信息http://ip:port/index/type/doc_id：查询指定的文档信息POST请求：http://ip:port/index/type/_search：查询文档，可以在请求体中添加json字符串来代表查询条件http://ip:port/index/type/doc_id/_update：修改文档，可以在请求体中指定json字符串代表修改的具体信息PUT请求：http://ip:port/index：创建一个索引，需要在请求体中指定索引的信息、类型和结构http://ip:port/index/type/_mappings：代表创建索引时，指定索引文档存储的属性信息DELETE请求：http://ip:port/index：删除跑路http://ip:port/index/type/doc_id：删除指定的文档3.3索引的操作3.3.1创建索引#创建索引PUT/person{"settings":{"number_of_shards":5,"number_of_replicas":1}}3.3.2查看索引信息#查看索引信息GET/person3.3.3删除索引#删除索引DELETE/person3.4ES中Field可以指定的类型官方类型字符串类型text：一般被用于全文检索，将当前Field进行分词。keyword：当前Field不会被分词。数值类型：long:integer：short：byte：double：float：half_float：精度比float小一半。scaled_float：根据一个long和scaled来表达一个浮点型。时间类型：date类型：针对时间类型指定具体的格式。布尔类型：boolean：表达true和false。二进制类型：binary类型暂时支持Base64encodestring范围类型：https://www.elastic.co/guide/en/elasticsearch/reference/6.5/range.html经纬度类型：https://www.elastic.co/guide/en/elasticsearch/reference/6.5/geo-point.html3.5创建索引并指定数据结构#创建索引，指定数据结构PUT/book#索引名{"settings":{"number_of_shards":5,#分片数"number_of_replicas":1#备份数},#指定数据结构"mappings":{"novel":{#类型type#文档存储的Field"properties":{#Field属性名"name":{"type":"text",#指定类型"analyzer":"ik_max_word",#指定类型"index":true,#指定当前的Field可以作为查询的条件"store":false#是否需要额外存储},"author":{"type":"keyword"},"count":{"type":"long"},"on-sale":{"type":"date","format":"yyyy-MM-ddHH:mm:ss||yyyy-MM-dd||epoch_millis"},"descr":{"type":"text","analyzer":"ik_max_word"}}}}}3.6文档的操作文档在ES服务中的唯一标识，_index，_type，_id三个个内容为组合，锁定一个文档。3.6.1新建文档自动生成_id#添加文档，自动生成idPOST/book/novel{"name":"盘龙","author":"whoami","count":100000,"on-sale":"2000-01-01","descr":"哈哈哈哈哈哈"}手动指定id#添加文档，手动指定idPUT/book/novel/1{"name":"红楼梦","author":"曹雪芹","count":10000000,"on-sale":"1985-01-01","descr":"哈哈哈哈哈哈嘻嘻嘻嘻嘻"}3.6.2修改文档覆盖式修改PUT/book/novel/1{"name":"红楼梦","author":"曹雪芹","count":34344554,"on-sale":"1985-01-01","descr":"哈哈哈哈哈哈嘻嘻嘻嘻嘻"}doc修改方式#修改文档，基于doc方式POST/book/novel/1/_update{"doc":{#指定需要修改的field和对应的值即可"count":1234567}}3.6.3删除文档#根据id删除文档DELETE/book/novel/_id四.Java操作elasticsearch4.1测试连接ES创建Maven工程导入依赖<dependencies><dependency><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId><version>6.5.4</version></dependency><dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId><version>6.5.4</version></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.16.22</version></dependency></dependencies>创建测试类，连接ESpublicclassESClient{publicstaticRestHighLevelClientgetClient(){//创建HttpHost对象HttpHosthttpHost=newHttpHost("192.168.0.17",9200);//创建RestClientBuilderRestClientBuilderclientBuilder=RestClient.builder(httpHost);//创建RestHighLevelClientRestHighLevelClientclient=newRestHighLevelClient(clientBuilder);//返回returnclient;}}publicclassDemo1{publicstaticvoidmain(String[]args){RestHighLevelClientclient=ESClient.getClient();System.out.println("OK");}}4.2Java创建索引publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();Stringindex="person";Stringtype="man";//1.准备关于索引的settingsSettings.Buildersettings=Settings.builder().put("number_of_shards",3).put("number_of_replicas",1);//2.准备关于索引的结构mappingsXContentBuildermappings=JsonXContent.contentBuilder().startObject().startObject("properties").startObject("name").field("type","text").endObject().startObject("age").field("type","integer").endObject().startObject("birthday").field("type","date").field("format","yyyy-MM-dd").endObject().endObject().endObject();//3.将settings和mappings封装到一个Request对象CreateIndexRequestrequest=newCreateIndexRequest(index).settings(settings).mapping(type,mappings);//4.通过client对象去连接ES并执行创建索引CreateIndexResponseresponse=client.indices().create(request,RequestOptions.DEFAULT);//5.输出测试System.out.println(request.toString());}4.2.1检查索引是否存在,删除索引检查索引是否存在publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();Stringindex="person";//1.准备request对象GetIndexRequestrequest=newGetIndexRequest();request.indices(index);//2.通过client去操作booleanexists=client.indices().exists(request,RequestOptions.DEFAULT);//3.输出System.out.println(exists);}删除索引publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();Stringindex="person";//1.准备request对象DeleteIndexRequestrequest=newDeleteIndexRequest();request.indices(index);//2.通过client去操作AcknowledgedResponsedelete=client.indices().delete(request,RequestOptions.DEFAULT);//3.输出System.out.println(delete.isAcknowledged());}4.3Java操作文档4.3.1添加文档publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="person";Stringtype="man";//1.准备一个json数据Personperson=newPerson(1,"张三",23,newDate());Stringjson=mapper.writeValueAsString(person);//2.准备一个request对象（手动指定id）IndexRequestrequest=newIndexRequest(index,type,person.getId().toString());request.source(json,XContentType.JSON);//3.通过client对象执行添加IndexResponseresp=client.index(request,RequestOptions.DEFAULT);//4.输出返回结果System.out.println(resp.getResult().toString());}4.3.2修改文档publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();Stringindex="person";Stringtype="man";StringdocId="1";//1.创建一个Map，指定需要修改的内容Map<String,Object>doc=newHashMap<String,Object>();doc.put("name","张大帅");//2.创建热区对象，封装数据UpdateRequestrequest=newUpdateRequest(index,type,docId);request.doc(doc);//3.通过client对象执行操作UpdateResponseupdate=client.update(request,RequestOptions.DEFAULT);//4.输出返回结果System.out.println(update.getResult().toString());}4.3.3删除文档publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();Stringindex="person";Stringtype="man";//1.封装Request对象DeleteRequestrequest=newDeleteRequest(index,type,"1");//2.client执行DeleteResponseresponse=client.delete(request,RequestOptions.DEFAULT);//3。输出结果System.out.println(response.getResult().toString());}4.4Java批量操作文档4.4.1批量添加文档publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="person";Stringtype="man";//1.准备多个json数据Personp1=newPerson(1,"张三",23,newDate());Personp2=newPerson(2,"李四",24,newDate());Personp3=newPerson(3,"王五",25,newDate());Stringjson1=mapper.writeValueAsString(p1);Stringjson2=mapper.writeValueAsString(p2);Stringjson3=mapper.writeValueAsString(p3);//2.创建Request，将准备好的数据封装进去BulkRequestrequest=newBulkRequest();request.add(newIndexRequest(index,type,p1.getId().toString()).source(json1,XContentType.JSON));request.add(newIndexRequest(index,type,p2.getId().toString()).source(json2,XContentType.JSON));request.add(newIndexRequest(index,type,p3.getId().toString()).source(json3,XContentType.JSON));//3.用client执行BulkResponseresp=client.bulk(request,RequestOptions.DEFAULT);//4.输出结果System.out.println(resp.toString());}4.4.2批量删除文档publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();Stringindex="person";Stringtype="man";//1.封装request对象BulkRequestrequest=newBulkRequest();request.add(newDeleteRequest(index,type,"1"));request.add(newDeleteRequest(index,type,"2"));request.add(newDeleteRequest(index,type,"3"));//2.执行client操作BulkResponseresp=client.bulk(request,RequestOptions.DEFAULT);//3.输出结果System.out.println(resp);}五.ES练习索引：sms-logs-index类型：sms-logs-type字段名称备注createDate创建时间sendDate发送时间longCode发送长号码，如"1069886622"mobile如：13800000000corpName发送公司名称，需要分词检索smsContent下发短信内容，需要分词检索state短信下发状态0成功1失败operatorId运营商编号1移动2联通3电信proince省份IpAddr下发服务器IP地址replyTotal短信状态报告返回时长（秒）fee扣费（分）六.ES的各种查询@Data@NoArgsConstructor@AllArgsConstructorpublicclassSmsLogs{privateStringid;//唯一IDprivateDatecreateDate;//创建时间privateDatesendDate;//发送时间privateStringlongCode;//发送的长号码privateStringmobile;//下发手机号privateStringcorpName;//发送公司名称privateStringsmsContent;//下发短信内容privateIntegerstate;//短信下发状态0成功1失败privateIntegeroperatorId;//运营商编号1移动2联通3电信privateStringprovince;//省份privateStringipAddr;//下发服务器IP地址privateIntegerreplyTotal;//短信状态报告返回时长privateIntegerfee;//费用}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.准备关于索引的settingsSettings.Buildersettings=Settings.builder().put("number_of_shards",3).put("number_of_replicas",1);//2.准备关于索引的结构mappingsXContentBuildermappings=JsonXContent.contentBuilder().startObject().startObject("properties").startObject("corpName").field("type","keyword").endObject().startObject("createDate").field("type","date").endObject().startObject("fee").field("type","long").endObject().startObject("ipAddr").field("type","ip").endObject().startObject("longCode").field("type","keyword").endObject().startObject("mobile").field("type","keyword").endObject().startObject("operatorId").field("type","integer").endObject().startObject("province").field("type","keyword").endObject().startObject("replyTotal").field("type","integer").endObject().startObject("sendDate").field("type","date").endObject().startObject("smsContent").field("type","text").field("analyzer","ik_max_word").endObject().startObject("state").field("type","integer").endObject().endObject().endObject();//3.将settings和mappings封装到一个Request对象CreateIndexRequestrequest=newCreateIndexRequest(index).settings(settings).mapping(type,mappings);//4.通过client对象去连接ES并执行创建索引CreateIndexResponseresponse=client.indices().create(request,RequestOptions.DEFAULT);//5.输出测试System.out.println(request.toString());}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.准备多个json数据SmsLogss1=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","盒马鲜生","盒马鲜生盒马鲜生盒马鲜生盒马鲜生",0,2,"上海","10.126.2.9",15,5);SmsLogss2=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","嘀嘀打车","嘀嘀打车嘀嘀打车嘀嘀打车嘀嘀打车",0,3,"北京","10.126.2.10",15,5);SmsLogss3=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","中国移动","中国移动中国移动中国移动中国移动",0,1,"沈阳","10.126.2.11",14,5);SmsLogss4=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","招商银行","招商银行招商银行招商银行招商银行",0,2,"大连","10.126.2.9",15,5);SmsLogss5=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","途虎养车","途虎养车途虎养车途虎养车途虎养车",0,2,"重庆","10.126.2.9",13,5);SmsLogss6=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","建设银行","建设银行建设银行建设银行建设银行",0,2,"鄂尔多斯","10.126.2.9",13,5);SmsLogss7=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","中国移动","中国移动中国移动中国移动中国移动",0,2,"赤峰","10.126.2.9",15,5);SmsLogss8=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","盒马鲜生","盒马鲜生盒马鲜生盒马鲜生盒马鲜生",0,2,"武汉","10.126.2.9",17,5);SmsLogss9=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","嘀嘀打车","嘀嘀打车嘀嘀打车嘀嘀打车嘀嘀打车",0,2,"上海","10.126.2.9",12,5);SmsLogss10=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","途虎养车","途虎养车途虎养车途虎养车途虎养车",0,2,"沈阳","10.126.2.9",11,5);SmsLogss11=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","建设银行","建设银行建设银行建设银行建设银行",0,2,"大连","10.126.2.9",10,5);SmsLogss12=newSmsLogs("",newDate(),newDate(),"10660000988","13800000000","嘀嘀打车","嘀嘀打车嘀嘀打车嘀嘀打车嘀嘀打车",0,2,"鄂尔多斯","10.126.2.9",19,5);Stringjson1=mapper.writeValueAsString(s1);Stringjson2=mapper.writeValueAsString(s2);Stringjson3=mapper.writeValueAsString(s3);Stringjson4=mapper.writeValueAsString(s4);Stringjson5=mapper.writeValueAsString(s5);Stringjson6=mapper.writeValueAsString(s6);Stringjson7=mapper.writeValueAsString(s7);Stringjson8=mapper.writeValueAsString(s8);Stringjson9=mapper.writeValueAsString(s9);Stringjson10=mapper.writeValueAsString(s10);Stringjson11=mapper.writeValueAsString(s11);Stringjson12=mapper.writeValueAsString(s12);//2.创建Request，将准备好的数据封装进去BulkRequestrequest=newBulkRequest();request.add(newIndexRequest(index,type,"1").source(json1,XContentType.JSON));request.add(newIndexRequest(index,type,"2").source(json2,XContentType.JSON));request.add(newIndexRequest(index,type,"3").source(json3,XContentType.JSON));request.add(newIndexRequest(index,type,"4").source(json4,XContentType.JSON));request.add(newIndexRequest(index,type,"5").source(json5,XContentType.JSON));request.add(newIndexRequest(index,type,"6").source(json6,XContentType.JSON));request.add(newIndexRequest(index,type,"7").source(json7,XContentType.JSON));request.add(newIndexRequest(index,type,"8").source(json8,XContentType.JSON));request.add(newIndexRequest(index,type,"9").source(json9,XContentType.JSON));request.add(newIndexRequest(index,type,"10").source(json10,XContentType.JSON));request.add(newIndexRequest(index,type,"11").source(json11,XContentType.JSON));request.add(newIndexRequest(index,type,"12").source(json12,XContentType.JSON));//3.用client执行BulkResponseresp=client.bulk(request,RequestOptions.DEFAULT);//4.输出结果System.out.println(resp.toString());}6.1term&terms查询6.1.1term查询javaterm查询是代表完全匹配，搜索之前不会对你的搜索关键字进行分词，直接去文档分词库中去匹配内容。#term查询POST/sms-logs-index/sms-logs-type/_search{"from":0,"size":5,"query":{"term":{"province":{"value":"北京"}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.from(0);builder.size(5);builder.query(QueryBuilders.termQuery("province","北京"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.1.2terms查询terms查询的机制跟term是一样的。terms是针对一个字段包含多个值的时候用。term：whereprovince=北京；terms：whereprovince=北京orprovince=？orprovince=？#terms查询POST/sms-logs-index/sms-logs-type/_search{"query":{"terms":{"province":["北京","上海"]}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.封装查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.termsQuery("province","北京","上海"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.2match查询match查询属于高层查询，他会根据你查询的字段类型不一样，采用不同的查询方式。查询的是日期或者是数值的话，它会将你基于的字符串查询内容转换为日期或者数值对待。查询的内容是一个不能被分词的内容（keyword），match查询不会对你指定的查询关键字进行分词。如果是可以被分词的内容（text），match会将你指定的查询内容根据一定的方式分词，去分词库中匹配指定的内容。match查询，实际底层是多个term查询，将多个term查询的结果给你封装到一起。6.2.1match_all查询查询全部内容。不指定任何查询条件。#match_all查询POST/sms-logs-index/sms-logs-type/_search{"query":{"match_all":{}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.matchAllQuery());builder.size(20);//ES默认查10条数据request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.2.2match查询指定一个Field作为筛选的条件#match查询POST/sms-logs-index/sms-logs-type/_search{"query":{"match":{"smsContent":"嘀嘀打车建设银行"}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.matchQuery("smsContent","嘀嘀打车建设银行"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.2.3布尔match查询基于一个Field匹配的内容，采用and或者or的方式连接#布尔match查询POST/sms-logs-index/sms-logs-type/_search{"query":{"match":{"smsContent":{"query":"嘀嘀打车","operator":"and"}}}}#布尔match查询POST/sms-logs-index/sms-logs-type/_search{"query":{"match":{"smsContent":{"query":"嘀嘀建设","operator":"or"}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.matchQuery("smsContent","嘀嘀建设").operator(Operator.OR));//Operator.ANDrequest.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.2.4multi_match查询match针对一个field做检索，multi_match针对多个field进行检索，多个field对应一个text。#multi_match查询POST/sms-logs-index/sms-logs-type/_search{"query":{"multi_match":{"query":"银行",#指定text"fields":["smsContent","province"]#指定field们}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.multiMatchQuery("银行","province","smsContent"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.3其他查询6.3.1id查询#id查询GET/sms-logs-index/sms-logs-type/_id#最后面添加要查询的idpublicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象GetRequestrequest=newGetRequest(index,type,"3");//2.执行查询GetResponseresp=client.get(request,RequestOptions.DEFAULT);//3.获取数据System.out.println(resp.getSourceAsMap());}6.3.2ids查询根据多个id查询，类似于MySQL中的whereidin(id1,id2,....)#ids查询POST/sms-logs-index/sms-logs-type/_search{"query":{"ids":{"values":["1","2","3"]}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.idsQuery().addIds("1","2","3"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.3.3prefix查询前缀查询，可以通过一个关键字去指定一个Field的前缀，从而查询到指定的文档。#prefix查询POST/sms-logs-index/sms-logs-type/_search{"query":{"prefix":{"corpName":{"value":"中国"}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.prefixQuery("corpName","中国"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.3.4fuzzy查询模糊查询，我们输入字符的大概，ES就可以去根据输入的内容大概去匹配一下结果。#fuzzy查询POST/sms-logs-index/sms-logs-type/_search{"query":{"fuzzy":{"corpName":{"value":"建设银航","prefix_length":2　　#指定前面几个字符是不允许出现错误}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.fuzzyQuery("corpName","建设银航").prefixLength(2));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.3.5wildcard查询通配查询,和MySQL中的like是一个套路,可以在查询时,在字符串中指定通配符*和占位符?#wildcard查询POST/sms-logs-index/sms-logs-type/_search{"query":{"wildcard":{"corpName":{"value":"嘀嘀*"}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.wildcardQuery("corpName","嘀嘀*"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.3.6range查询范围查询,只针对数值类型,对某一个Field进行大于或者小于的范围#range查询POST/sms-logs-index/sms-logs-type/_search{"query":{"range":{"fee":{"gte":5,"lte":10}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.rangeQuery("fee").gte(5).lte(10));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.3.7regexp查询正则查询,通过你编写的正则表达式去匹配内容.Ps:prefix,fuzzy,wildcard和regexp查询效率相对比较低,要求效率高时,避免使用#regexp查询POST/sms-logs-index/sms-logs-type/_search{"query":{"regexp":{"mobile":"180[0-9]{8}"}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.regexpQuery("mobile","180[0-9]{8}"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.4深分页ScrollES对from+size是有限制的,from和size二者之和不能超过1W。原理：form+size在ES查询数据的方式：第一步，将用户指定的关键字进行分词。第二步，将词汇去分词库中进行检索，得到多个文档的id。第三步，去各个分片中拉取指定的数据，耗时较长。第四步，将数据根据score进行排序，耗时较长。第五步，根据from的值，将查询到的数据舍弃一部分。第六步，返回结果。Scroll+size在ES查询数据的方式：第一步，将用户指定的关键字进行分词。第二步，将词汇去分词库中检索，得到多个文档的id。第三步，将文档的id存放在ES的上下文中。第四步，根据指定的size去ES中检索指定的数据，拿完数据文档id，会从上下文中移除。第五步，如果需要下一页数据，直接去ES的上下文中找后续的内容。Scroll的弊端：不适合做实时查询。#执行Scroll查询，返回第一页数据，并且将文档id信息存放在ES上下文中，指定生存时间1分钟POST/sms-logs-index/sms-logs-type/_search?scroll=1m{"query":{"match_all":{}},"size":2,"sort":[{"fee":{"order":"desc"}}]}#根据scroll查询下一页数据POST/_search/scroll{"scroll_id":"根据第一步得到的scroll_id去指定","scroll":"1m"}#删除scroll在ES上下文中的数据DELETE/_search/scroll/scroll_idpublicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定scroll信息request.scroll(TimeValue.timeValueMinutes(1L));//3.指定查询的条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.size(4);builder.sort("fee",SortOrder.DESC);builder.query(QueryBuilders.matchAllQuery());request.source(builder);//4.获取数据scroll_id，sourceSearchResponseresp=client.search(request,RequestOptions.DEFAULT);StringscrollId=resp.getScrollId();System.out.println("----------首页-----------");for(SearchHithit:resp.getHits().getHits()){System.out.println(hit.getSourceAsMap());}while(true){//5.循环-创建SearchScrollRequestSearchScrollRequestscrollRequest=newSearchScrollRequest(scrollId);//6.指定scrollId的生存时间scrollRequest.scroll(TimeValue.timeValueMinutes(1L));//7.执行查询获取返回结果SearchResponsescrollResp=client.scroll(scrollRequest,RequestOptions.DEFAULT);//8.判断是否查询到了数据SearchHit[]hits=scrollResp.getHits().getHits();if(hits!=null&&hits.length>0){System.out.println("----------下一页-----------");for(SearchHithit:hits){System.out.println(hit.getSourceAsMap());}}else{//9.判断没有查询到数据-退出循环System.out.println("----------结束-----------");break;}}//10.创建ClearScrollRequestClearScrollRequestclearScrollRequest=newClearScrollRequest();//11.指定scrollIdclearScrollRequest.addScrollId(scrollId);//12.删除ScrollIdClearScrollResponseclearScrollResponse=client.clearScroll(clearScrollRequest,RequestOptions.DEFAULT);//13.输出结果System.out.println("删除scroll："+clearScrollResponse.isSucceeded());}6.5delete-by-query根据term、match等查询方式去删除大量文档Ps：如果需要删除的内容是index下的大部分数据，不推荐此方法，推荐创建一个全新的index，将保留的文档内容，添加到全新的索引中#delete-by-queryPOST/sms-logs-index/sms-logs-type/_delete_by_query{"query":{"range":{"fee":{"lt":5}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象DeleteByQueryRequestrequest=newDeleteByQueryRequest(index);request.types(type);//2.指定查询条件request.setQuery(QueryBuilders.rangeQuery("fee").lt(5));//3.执行查询BulkByScrollResponseresp=client.deleteByQuery(request,RequestOptions.DEFAULT);//4.获取数据System.out.println(resp.toString());}6.6复合查询6.6.1bool查询复合过滤器，将你的多个查询条件，以一定的逻辑组合在一起。must：所有的条件用must组合在一起，表示And的意思。must_not：将must_not中的条件，全部都不匹配，表示Not的意思。should：所有的条件用should组合在一起，表示or的意思。#查询省份为武汉或北京#运营商不是联通#smsContent中包含嘀嘀和打车#bool查询POST/sms-logs-index/sms-logs-type/_search{"query":{"bool":{"should":[{"term":{"province":{"value":"北京"}}},{"term":{"province":{"value":"武汉"}}}],"must_not":[{"term":{"operatorId":{"value":"2"}}}],"must":[{"match":{"smsContent":"嘀嘀"}},{"match":{"smsContent":"打车"}}]}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();BoolQueryBuilderboolQuery=QueryBuilders.boolQuery();//查询省份为武汉或者北京boolQuery.should(QueryBuilders.termQuery("province","北京"));boolQuery.should(QueryBuilders.termQuery("province","武汉"));//运营商不是联通boolQuery.mustNot(QueryBuilders.termQuery("operatorId",2));//smsContent中包含嘀嘀和打车boolQuery.must(QueryBuilders.matchQuery("smsContent","嘀嘀"));boolQuery.must(QueryBuilders.matchQuery("smsContent","打车"));builder.query(boolQuery);request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){System.out.println(hit.getSourceAsMap());}}6.6.2boosting查询boosting查询可以帮助我们去影响查询后的score。positive：只有匹配上positive的查询的内容，才会被放到返回的结果集中。negative：如果匹配上positive并且也匹配上了negative，就可以降低这样文档的score。negative_boost：指定系数，必须小于1.0。关于查询时，分数是如何计算的：搜索的关键字在文档中出现的频次越高，分数就越高。指定的文档内容越短，分数越高。在搜索时，指定的关键字也会被分词，这个被分词的内容，在分词库中匹配的个数越多，分数越高。#boosting查询POST/sms-logs-index/sms-logs-type/_search{"query":{"boosting":{"positive":{"match":{"smsContent":"嘀嘀打车"}},"negative":{"match":{"smsContent":"打车"#在positive匹配的基础上再去匹配positive匹配得到的内容}},"negative_boost":0.5}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();BoostingQueryBuilderboostingQuery=QueryBuilders.boostingQuery(QueryBuilders.matchQuery("smsContent","嘀嘀打车"),QueryBuilders.matchQuery("smsContent","打车")).negativeBoost(0.5f);builder.query(boostingQuery);request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.7filter查询query查询，根据查询的条件，去计算文档的匹配度得到一个分数，并且根据分数进行排序，不会做缓存。filter查询，根据查询的条件去查询文档，但不会去计算分数，而且filter会对经常被过滤的数据进行缓存。#filter查询POST/sms-logs-index/sms-logs-type/_search{"query":{"bool":{"filter":[{"term":{"corpName":"途虎养车"}},{"range":{"fee":{"lte":5}}}]}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();BoolQueryBuilderboolQuery=QueryBuilders.boolQuery();boolQuery.filter(QueryBuilders.termQuery("corpName","途虎养车"));boolQuery.filter(QueryBuilders.rangeQuery("fee").lte(5));builder.query(boolQuery);request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){Map<String,Object>result=hit.getSourceAsMap();System.out.println(result);}}6.8高亮查询高亮查询就是用户输入的关键字，以一定的特殊样式展示给用户。ES提供了一个highlight属性，和query同级别。fragment_size：指定高亮数据展示多少个字符。pre_tags：指定前缀标签，例如：post_tags：指定后缀标签，例如：fields：指定哪几个Fields一高亮的形式返回publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.query(QueryBuilders.matchQuery("smsContent","嘀嘀"));HighlightBuilderhighlightBuilder=newHighlightBuilder();highlightBuilder.field("smsContent",6).preTags("<fontcolor='red'>").postTags("</font>");builder.highlighter(highlightBuilder);request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){System.out.println(hit.getHighlightFields().get("smsContent"));}}6.9聚合查询ES的聚合查询和MySQL的聚合查询类似，ES的聚合查询相比MySQL强大的多，ES提供的统计数据的方式多得多。#ES的聚合查询的RESTful语法POST/index/type/_search{"aggs":{"名字(agg)":{"agg_type":{"属性":"值"}}}}6.9.1去重计数查询去重计数，即Cardinality，将返回的文档中的一个指定的field进行去重，统计一共有多少条。POST/sms-logs-index/sms-logs-type/_search{"aggs":{"agg":{"cardinality":{"field":"province"}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.aggregation(AggregationBuilders.cardinality("agg").field("province"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据Cardinalityagg=resp.getAggregations().get("agg");System.out.println(agg.getValue());}6.9.2范围统计统计一定范围内出现的文档个数。范围统计可以针对普通的数值、时间类型、IP类型都可以统计。数值统计POST/sms-logs-index/sms-logs-type/_search{"aggs":{"a":{"range":{"field":"fee","ranges":[{"to":5},{"from":5,#from包含等于"to":10},{"from":10}]}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.aggregation(AggregationBuilders.range("agg").field("fee").addUnboundedTo(5).addRange(5,10).addUnboundedFrom(10));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据Rangeagg=resp.getAggregations().get("agg");for(Range.Bucketbucket:agg.getBuckets()){Stringkey=bucket.getKeyAsString();Objectfrom=bucket.getFrom();Objectto=bucket.getTo();longdocCount=bucket.getDocCount();System.out.println(String.format("key：%s，form：%s，to：%s，docCount：%s",key,from,to,docCount));}}时间范围统计POST/sms-logs-index/sms-logs-type/_search{"aggs":{"agg":{"date_range":{"field":"createDate","format":"yyyy","ranges":[{"to":2000},{"from":2000}]}}}}IP范围统计POST/sms-logs-index/sms-logs-type/_search{"aggs":{"agg":{"ip_range":{"field":"ipAddr","ranges":[{"to":"10.126.2.10"},{"from":"10.126.2.10"}]}}}}6.9.3统计聚合查询它可以帮你指定Field的最大值、最小值、平均值。。。。使用：extended_statsPOST/sms-logs-index/sms-logs-type/_search{"aggs":{"agg":{"extended_stats":{"field":"fee"}}}}publicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="sms-logs-index";Stringtype="sms-logs-type";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();builder.aggregation(AggregationBuilders.extendedStats("agg").field("fee"));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据ExtendedStatsagg=resp.getAggregations().get("agg");System.out.println(agg.getCount());System.out.println(agg.getMin());System.out.println(agg.getMax());System.out.println(agg.getAvg());System.out.println(agg.getSum());}6.10地图经纬度搜索ES中提供了一个数据类型geo_point,这个类型就是用来存储经纬度的。先创建测试数据PUT/map{"settings":{"number_of_shards":5,"number_of_replicas":1},"mappings":{"map":{"properties":{"name":{"type":"text"},"location":{"type":"geo_point"}}}}}PUT/map/map/1{"name":"天安门","location":{"lon":116.403981,"lat":39.914492}}PUT/map/map/2{"name":"海淀公园","location":{"lon":116.302509,"lat":39.991152}}PUT/map/map/3{"name":"北京动物园","location":{"lon":116.343184,"lat":39.947468}}6.10.1ES的地图检索方式geo_distance：直线距离检索方式geo_bounding_box：以两个点确定一个矩形，获取在矩形内的全部数据geo_polygon：以多个点，确定一个多边形，获取多边形内的全部数据6.10.2基于RESTful实现地图检索geo_distancePOST/map/map/_search{"query":{"geo_distance":{"location":{#确定一个点"lon":116.433733,"lat":39.908404},"distance":3000,#确定半径"distance_type":"arc"#指定形状为圆形}}}geo_bounding_boxPOST/map/map/_search{"query":{"geo_bounding_box":{"location":{"top_left":{#左上角坐标点"lon":116.326943,"lat":39.95499},"bottom_right":{#右下角坐标点"lon":116.347783,"lat":39.939281}}}}}geo_polygonPOST/map/map/_search{"query":{"geo_polygon":{"location":{"points":[#指定多个点确定一个多边形{"lon":116.298772,"lat":39.998227},{"lon":116.295394,"lat":39.975672},{"lon":116.323206,"lat":39.989936}]}}}}6.10.3Java实现geo_polygonpublicstaticvoidmain(String[]args)throwsIOException{RestHighLevelClientclient=ESClient.getClient();ObjectMappermapper=newObjectMapper();Stringindex="map";Stringtype="map";//1.创建request对象SearchRequestrequest=newSearchRequest(index);request.types(type);//2.指定查询条件SearchSourceBuilderbuilder=newSearchSourceBuilder();List<GeoPoint>points=newArrayList<GeoPoint>();points.add(newGeoPoint(39.998227,116.298772));points.add(newGeoPoint(39.975672,116.295394));points.add(newGeoPoint(39.989936,116.323206));builder.query(QueryBuilders.geoPolygonQuery("location",points));request.source(builder);//3.执行查询SearchResponseresp=client.search(request,RequestOptions.DEFAULT);//4.获取数据for(SearchHithit:resp.getHits().getHits()){System.out.println(hit.getSourceAsMap());}}

2021-01-21 21:31 · ElasticSearch

[文章] Redis的安装以及相关操作

一.Redis安装1.1安装Redisversion:'3.1'services:redis:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redisenvironment:-TZ=Asia/Shanghaiports:-6379:6379二.Redis常用的命令2.1Redis存储数据的结构常用的5种数据结构：key-string：一个key对应一个值。一般用于存储一个值。key-hash：一个key对应一个Map。存储一个对象数据。key-list：一个key对应一个列表。使用list结构实现栈结构和队列结构。key-set：一个key对应一个集合。交集，差集和并集的操作。key-zset：一个key对应一个有序的集合。排行榜或积分存储等操作。另外3种数据结构：HyperLogLog：计算近似值。GEO：存储地理位置。BIT：一般存储的也是一个字符串，存储的是一个byte[]。2.2string结构的常用命令#1.添加值(如果key已经持有其他值，SET就覆写旧值，无视类型)setkeyvalue#2.取值getkey#3.批量操作msetkey1value1key2value2..keyNvalueNmgetkey1key2..keyN#4.自增命令(+1)incrkey#5.自减命令(-1)decrkey#6.自增或自减指定数量incrbykeynumdecrbykeynum#7.设置值的同时，指定生存时间setexkeysecondvalue#8.如果key存在，什么都不做，不存在的话，和set命令一样setnxkeyvalue#9.在key对应的value后，追加内容appendkeyvalue#10.查看value字符串的长度strlenkey2.3hash常用命令#1.存储数据hsetkeyfieldvalue#2.获取数据hgetkeyfield#3.批量操作hmsetkeyfield1value1field2value2..fieldNvalueNhmgetkeyfield1field2..fieldNRedis其它结构命令2.4key的常用命令#1.查看Redis中的全部的key(pattern:*,xxx*,*xxx)keyspattern#2.查看某一个key是否存在(1->存在，0->不存在)existskey#3.删除keydelkey1key2..keyN#4.设置key的生存时间，单位为秒，或毫秒，设置能活多久expirekeysecondpexpirekeymillisrcond#5.设置key的生存时间，单位为秒，或毫秒，设置活到哪一个时间点expireatkeytimestamppexpireatkeymillisrcond#6.查看key的剩余生存时间，单位为秒，或毫秒(-2->当前key不存在，-1->当前key没有设置生存时间，具体的剩余生存时间)ttlkeypttlkey#7.移除key的生存时间(1-移除成功，0-key不存在生存时间，key不存在)persistkey#8.移动key到另一个库中movekeydb2.5库的操作命令#1.选择操作库select0~15#2.清空当前库flushdb#3.清空所有库flushall#4.查询当前库有多少个keydbsize#5.最后一次操作的时间lastsave#6.实时监控Redis服务接受到的命令monitor三.Java连接Redis3.1Jedis连接Redis创建maven项目导入需要的依赖<dependencies><dependency><groupId>redis.clients</groupId><artifactId>jedis</artifactId><version>2.9.0</version></dependency><dependency><groupId>junit</groupId><artifactId>junit</artifactId><version>4.12</version><scope>test</scope></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.16.20</version></dependency></dependencies>测试//1.连接RedisJedisjedis=newJedis("xxx.xxx.xxx.xxx",6379);//2.操作Redisjedis.set("name","李四");//3.释放资源jedis.close();3.2Jedis如何存储|获取一个对象,以byte[]的形式准备一个实体类@Data@NoArgsConstructor@AllArgsConstructorpublicclassUserimplementsSerializable{privateIntegerid;privateStringname;privateDatebirthday;}导入spring-context依赖，以方便对象在字节数组之间转换<dependency><groupId>org.springframework</groupId><artifactId>spring-context</artifactId><version>4.3.18.RELEASE</version></dependency>以byte[]形式存储对象//存储对象-以byte[]形式存储在Redis中//1。连接RedisJedisjedis=newJedis("xxx.xxx.xxx.xxx",6379);//2.1准备key(String)-value(User)Stringkey="user";Uservalue=newUser(1,"张三",newDate());//2.2将key和value转换为byte[]byte[]byteKey=SerializationUtils.serialize(key);byte[]byteValue=SerializationUtils.serialize(value);//2.3将key和value存储到Redis中jedis.set(byteKey,byteValue);//3.释放资源jedis.close();以byte[]形式获取对象//获取对象-以byte[]形式在Redis中获取//1。连接RedisJedisjedis=newJedis("xxx.xxx.xxx.xxx",6379);//2.1准备keyStringkey="user";//2.2将key转换为byte[]byte[]byteKey=SerializationUtils.serialize(key);//2.3jedis去Redis中获取valuebyte[]byteValue=jedis.get(byteKey);//2.4将value反序列化为User对象Useruser=(User)SerializationUtils.deserialize(byteValue);//2.5输出System.out.println(user);//3.释放资源jedis.close();3.3Jedis如何存储|获取一个对象,以String的形式为了把对象转换成String形式，要导入fastjson依赖<dependency><groupId>com.alibaba</groupId><artifactId>fastjson</artifactId><version>1.2.47</version></dependency>以String形式存储对象//以String形式存储对象//1.连接RedisJedisjedis=newJedis("xxx.xxx.xxx.xxx",6379);//2.1准备key(String)-value(User)StringstringKey="stringUser";Uservalue=newUser(2,"李四",newDate());//2.2使用fastJSON将value转化为json字符串StringstringValue=JSON.toJSONString(value);//2.3存储到Redis中jedis.set(stringKey,stringValue);//3.释放资源jedis.close();以String形式获取对象//以String形式获取对象//1.连接RedisJedisjedis=newJedis("xxx.xxx.xxx.xxx",6379);//2.1准备keyStringstringKey="stringUser";//2.2jedis去Redis中获取valueStringvalue=jedis.get(stringKey);//2.3将value反序列化为User对象Useruser=JSON.parseObject(value,User.class);//2.4输出System.out.println(user);//3.释放资源jedis.close();3.4Jedis连接池的操作上面我们每操作一次都要创建连接，最后释放连接，如果还要操作的话还得做同样的操作，这会对性能有很大的影响。所以用连接池的话，对性能影响不大。//1.创建连接池JedisPoolpool=newJedisPool("xxx.xxx.xxx.xxx",6379);//2.通过连接池获取jedis对象Jedisjedis=pool.getResource();//3.操作Stringuser=jedis.get("xxx");//xxx为keySystem.out.println(user);//4.释放资源，把jedis对象还给连接池jedis.close();还可以设置连接池的基本参数//1.创建连接池的配置信息GenericObjectPoolConfigpoolConfig=newGenericObjectPoolConfig();poolConfig.setMaxTotal(100);//设置连接池的活跃数poolConfig.setMaxIdle(10);//最大空闲数poolConfig.setMinIdle(5);//最小空闲数poolConfig.setMaxWaitMillis(3000);//当连接池空了之后，多久没获取到Jedis对象，就超时，设置timeout时间//2.创建连接池JedisPoolpool=newJedisPool(poolConfig,"xxx.xxx.xxx.xxx",6379);//3.通过连接池获取jedis对象Jedisjedis=pool.getResource();//4.操作Stringuser=jedis.get("xxx");//xxx为keySystem.out.println(user);//5.释放资源，把jedis对象还给连接池jedis.close();四.Redis的管道操作假如客户端要给Redis服务器发送了三个命令，它会如下图一样分三批次执行，但是执行命令并反馈时网络延迟等等问题消耗的时间很大，所以Redis的管道操作就是解决这个问题的。管道操作就是一次性把所有要执行的命令捆在一起，给Redis服务器发过去。管道操作普通操作//1.创建连接池JedisPoolpool=newJedisPool("192.168.0.16",6379);longl=System.currentTimeMillis();//2.获取连接对象Jedisjedis=pool.getResource();//3.执行incr-100000次for(inti=0;i<100000;i++){jedis.incr("pp");}//4.释放资源jedis.close();System.out.println(System.currentTimeMillis()-l);管道操作//*************管道操作************//1.获取连接对象longl=System.currentTimeMillis();Jedisjedis=pool.getResource();//2.创建管道Pipelinepipeline=jedis.pipelined();//3.执行incr-100000次for(inti=0;i<100000;i++){pipeline.incr("qq");}//4.执行命令pipeline.syncAndReturnAll();//5.释放资源jedis.close();System.out.println(System.currentTimeMillis()-l);五.Redis其他配置及集群修改yml文件，以方便后期修改Redis配置信息version:'3.1'services:redis:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redisenvironment:-TZ=Asia/Shanghaiports:-6379:6379volumes:-./conf/redis.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]5.1Redis的AUTH我们之前都是直接连接Redis服务的，这样会不安全，这个AUTH相当于密码方式一：在redis.conf文件中添加以下配置信息就可以了啦#后面为密码requirepassadmin之后在Java连接Redis时也需要这个，只需添加如下代码jedis.auth("admin")如果用连接池的话可以运用以下构造函数publicJedisPool(GenericObjectPoolConfigpoolConfig,Stringhost,intport,inttimeout,Stringpassword)publicJedisPool(GenericObjectPoolConfigpoolConfig,Stringhost,intport,inttimeout,Stringpassword,intdatabase)方式二：在不修改redis.conf文件的前提下，在第一次连接Redis时，输入命令：configsetrequirepass密码，后续再次操作Redis时，需要先auth做一下校验5.2Redis的持久化机制5.2.1RDB持久化机制RDB是Redis默认的持久化机制，在指定的时间间隔内将内存中的数据集快照写入磁盘。RDB持久化是指在指定的时间间隔内将内存中的数据集快照写入磁盘，实际操作过程是fork一个子进程，先将数据集写入临时文件，写入成功后，再替换之前的文件，用二进制压缩存储。RDB持久化机制特点：RDB持久化文件，速度比较快，而且存储的是二进制的文件，传输起来很方便。RDB持久化的时机：save9001#900秒之内，有一个key改变了，就执行RDB持久化，以下都一个意思save30010save6010000RDB无法保证数据的绝对安全。#RDB持久化机制配置文件#代表RDB执行的时机save9001#900秒之内，有一个key改变了，就执行RDB持久化，以下都一个意思save30010save6010000#开启RDB持久化的压缩rdbcompressionyes#RDB持久化文件的名称dbfilenamedump.rdb5.2.2AOF持久化机制另一种持久化方式叫AOF，默认是关闭的，原理是将Reids的操作日志以追加的方式写入文件。AOF持久化以日志的形式记录服务器所处理的每一个写、删除操作，查询操作不会记录，以文本的方式记录，可以打开文件看到详细的操作记录。#AOF配置文件appendonlyyes#开启AOF持久化appendfilename"redis.aof"#AOF文件的名称#以下配置只能三选一appendfsyncalways#每次有数据修改发生时都会写入AOF文件。appendfsynceverysec#每秒钟同步一次，该策略为AOF的缺省策略。appendfsyncno#从不同步。高效但是数据不会被持久化。5.2.3二者优缺点RDB存在哪些优势呢？一旦采用该方式，那么你的整个Redis数据库将只包含一个文件，这对于文件备份而言是非常完美的。比如，你可能打算每个小时归档一次最近24小时的数据，同时还要每天归档一次最近30天的数据。通过这样的备份策略，一旦系统出现灾难性故障，我们可以非常容易的进行恢复。对于灾难恢复而言，RDB是非常不错的选择。因为我们可以非常轻松的将一个单独的文件压缩后再转移到其它存储介质上。性能最大化。对于Redis的服务进程而言，在开始持久化时，它唯一需要做的只是fork出子进程，之后再由子进程完成这些持久化的工作，这样就可以极大的避免服务进程执行IO操作了。相比于AOF机制，如果数据集很大，RDB的启动效率会更高。RDB又存在哪些劣势呢？如果你想保证数据的高可用性，即最大限度的避免数据丢失，那么RDB将不是一个很好的选择。因为系统一旦在定时持久化之前出现宕机现象，此前没有来得及写入磁盘的数据都将丢失。由于RDB是通过fork子进程来协助完成数据持久化工作的，因此，如果当数据集较大时，可能会导致整个服务器停止服务几百毫秒，甚至是1秒钟。AOF的优势有哪些呢？该机制可以带来更高的数据安全性，即数据持久性。Redis中提供了3中同步策略，即每秒同步、每修改同步和不同步。事实上，每秒同步也是异步完成的，其效率也是非常高的，所差的是一旦系统出现宕机现象，那么这一秒钟之内修改的数据将会丢失。而每修改同步，我们可以将其视为同步持久化，即每次发生的数据变化都会被立即记录到磁盘中。可以预见，这种方式在效率上是最低的。至于无同步，无需多言，我想大家都能正确的理解它。由于该机制对日志文件的写入操作采用的是append模式，因此在写入过程中即使出现宕机现象，也不会破坏日志文件中已经存在的内容。然而如果我们本次操作只是写入了一半数据就出现了系统崩溃问题，不用担心，在Redis下一次启动之前，我们可以通过redis-check-aof工具来帮助我们解决数据一致性的问题。如果日志过大，Redis可以自动启用rewrite机制。即Redis以append模式不断的将修改数据写入到老的磁盘文件中，同时Redis还会创建一个新的文件用于记录此期间有哪些修改命令被执行。因此在进行rewrite切换时可以更好的保证数据安全性。AOF包含一个格式清晰、易于理解的日志文件用于记录所有的修改操作。事实上，我们也可以通过该文件完成数据的重建。AOF的劣势有哪些呢？对于相同数量的数据集而言，AOF文件通常要大于RDB文件。RDB在恢复大数据集时的速度比AOF的恢复速度要快。根据同步策略的不同，AOF在运行效率上往往会慢于RDB。总之，每秒同步策略的效率是比较高的，同步禁用策略的效率和RDB一样高效。二者选择的标准，就是看系统是愿意牺牲一些性能，换取更高的缓存一致性（aof），还是愿意写操作频繁的时候，不启用备份来换取更高的性能，待手动运行save的时候，再做备份（rdb）。rdb这个就更有些eventuallyconsistent的意思了。不过生产环境其实更多都是二者结合使用的。如果同时开启了AOF和RDB持久化，那么在Redis宕机重启之后，需要加载一个持久化文件，优先选择AOF文件。如果先开启了RDB，再次开启AOF，如果RDB执行了持久化，那么RDB文件中的内容会被AOF覆盖掉。5.3Redis的事务Redis事务的本质是一组命令的集合。事务支持一次执行多个命令，一个事务中所有命令都会放在一个队列中。在事务执行过程，会按照顺序串行化执行队列中的命令，如果取消了事务，一个队列中的命令全部作废。其他客户端提交的命令请求不会插入到事务执行命令序列中。#开启事务命令multi#执行事务命令exec#取消事务命令discard如上图，开启事务后，执行了两个命令，返回QUEUED,表示把命令存到了队列之中，之后执行exec命令执行事务。Redis的事务想发挥功能，需要配合watch监听机制。在开启事务之前，先通过watch命令去监听一个或多个key，在开启事务之后，如果有其他客户端修改了监听的key，事务会自动取消。如果执行了事务，或者取消了事务，watch监听会自动消除，一般不需要手动执行unwatch命令。5.4Redis的主从架构单机版Redis存在读写瓶颈的问题，搭建主从架构会能解决这个问题。version:'3.1'services:redis1:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis1environment:-TZ=Asia/Shanghaiports:-7001:6379volumes:-./conf/redis1.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]redis2:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis2environment:-TZ=Asia/Shanghaiports:-7002:6379volumes:-./conf/redis2.conf:/usr/local/redis/redis.conflinks:-redis1:mastercommand:["redis-server","/usr/local/redis/redis.conf"]redis3:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis3environment:-TZ=Asia/Shanghaiports:-7003:6379volumes:-./conf/redis3.conf:/usr/local/redis/redis.conflinks:-redis1:mastercommand:["redis-server","/usr/local/redis/redis.conf"]从节点的配置replicaofmaster6379#master是links后面的别名5.5Redis的哨兵如果在Redis集群中master节点挂了，Redis的主从架构就崩了。哨兵就可以解决这个问题。master节点挂了，哨兵会从剩下的从节点中选出一个节点，让它当master节点。修改了docker-compose.yml，为了可以在容器内部使用哨兵的配置。version:'3.1'services:redis1:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis1environment:-TZ=Asia/Shanghaiports:-7001:6379volumes:-./conf/redis1.conf:/usr/local/redis/redis.conf-./conf/sentinel1.conf:/data/sentinel.conf#修改处command:["redis-server","/usr/local/redis/redis.conf"]redis2:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis2environment:-TZ=Asia/Shanghaiports:-7002:6379volumes:-./conf/redis2.conf:/usr/local/redis/redis.conf-./conf/sentinel2.conf:/data/sentinel.conf#修改处links:-redis1:mastercommand:["redis-server","/usr/local/redis/redis.conf"]redis3:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis3environment:-TZ=Asia/Shanghaiports:-7003:6379volumes:-./conf/redis3.conf:/usr/local/redis/redis.conf-./conf/sentinel3.conf:/data/sentinel.conf#修改处links:-redis1:mastercommand:["redis-server","/usr/local/redis/redis.conf"]准备哨兵的配置文件，并且在容器内部手动开启哨兵即可。#哨兵需要后台启动daemonizeyes#指定Master节点的ip和端口（主）sentinelmonitormasterlocalhost63792#指定Master节点的ip和端口（从）sentinelmonitormastermaster63792#哨兵每个多久监听一次redis架构sentineldown-after-millisecondsmymaster100005.6Redis的集群Redis集群在保证主从加哨兵的基本功能之外，还能够提升Redis存储数据的能力。这里从节点和上面的主从架构的从节点是不同的概念。这个从节点只管做备份的操作，不做读操作，如果他的配分数据的redis服务挂了，这个从节点就顶替他的位置准备一个docker-compose.yml文件version:"3.1"services:redis1:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis1environment:-TZ=Asia/Shanghaiports:-7001:7001-17001:17001volumes:-./conf/redis1.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]redis2:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis2environment:-TZ=Asia/Shanghaiports:-7002:7002-17002:17002volumes:-./conf/redis2.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]redis3:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis3environment:-TZ=Asia/Shanghaiports:-7003:7003-17003:17003volumes:-./conf/redis3.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]redis4:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis4environment:-TZ=Asia/Shanghaiports:-7004:7004-17004:17004volumes:-./conf/redis4.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]redis5:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis5environment:-TZ=Asia/Shanghaiports:-7005:7005-17005:17005volumes:-./conf/redis5.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]redis6:image:daocloud.io/library/redis:5.0.7restart:alwayscontainer_name:redis6environment:-TZ=Asia/Shanghaiports:-7006:7006-1700:17006volumes:-./conf/redis6.conf:/usr/local/redis/redis.confcommand:["redis-server","/usr/local/redis/redis.conf"]redis.conf配置文件，下面是redis1的配置文件，其它的修改一下就可以啦。#指定redis的端口号port7001#开启集群cluster-enabledyes#集群信息的文件cluster-config-filenodes-7001.conf#集群的对外ip地址#Linux主机的ip地址cluster-announce-ip192.168.0.17#Redis的端口cluster-announce-port7001#集群的总线端口cluster-announce-bus-port17001启动了6个Redis的节点。随便跳转到一个容器内部，使用redis-cli管理集群。redis-cli--clustercreate192.168.0.17:7001192.168.0.17:7002192.168.0.17:7003192.168.0.17:7004192.168.0.17:7005192.168.0.17:7006--cluster-replicas15.7Java连接Redis集群使用JedisCluster对象连接Redis集群//创建Set<HostAndPort>nodesSet<HostAndPort>nodes=newHashSet<HostAndPort>();nodes.add(newHostAndPort("192.168.0.17",7001));nodes.add(newHostAndPort("192.168.0.17",7002));nodes.add(newHostAndPort("192.168.0.17",7003));nodes.add(newHostAndPort("192.168.0.17",7004));nodes.add(newHostAndPort("192.168.0.17",7005));nodes.add(newHostAndPort("192.168.0.17",7006));//创建JedisCluster对象JedisClusterjedisCluster=newJedisCluster(nodes);//操作Stringvalue=jedisCluster.get("b");System.out.println(value);六.Redis的常见问题6.1key的生存时间到了，Redis会立即删除么？不会立即删除。定期删除：Redis每隔一段时间就会去查看Redis设置了过期时间的key，会在100ms的间隔中默认查看3个key。惰性删除：如果当你查询(get)一个已经过了生存时间的key时，Redis会先查看当前key的生存时间，是否已经到达了，直接删除当前key，并且给用户返回一个空值。6.2Redis的淘汰机制在Redis内存已经满的时候，添加了一个新的数据，就会执行淘汰机制。volatile-lru：在内存不足时，Redis会在设置了生存时间的key中干掉一个最近最少使用的key。allkeys-lru：在内存不足时，Redis会在全部的key中干掉一个最近最少使用的key。volatile-lfu：在内存不足时，Redis会在设置了生存时间的key中干掉一个最近最少频次使用的key。allkeys-lfu：在内存不足时，Redis会在全部的key中干掉一个最近最少频次使用的key。volatile-random：在内存不足时，Redis会在设置了生存时间的key中随机干掉一个key。allkeys-random：在内存不足时，Redis会在全部的key中随机干掉一个key。volatile-ttl：在内存不足时，Redis会在设置了生存时间的key中干掉一个剩余时间最少的key。noeviction：在内存不足时，直接报错。默认的配置指定淘汰机制的方式：maxmemory-policy具体机制指定Redis的最大内存：maxmemory

2021-01-21 21:17 · Redis

回到顶部