Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Contribute to GitLab
Sign in / Register
Toggle navigation
D
dsk-dsc-flink
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
shezaixing
dsk-dsc-flink
Commits
4fc840af
Commit
4fc840af
authored
Jan 13, 2024
by
shezaixing
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
upate
parent
a0b42df9
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
17 additions
and
1 deletion
+17
-1
SyncCustomerDataSource.java
...n/java/com/dsk/flink/dsc/sync/SyncCustomerDataSource.java
+1
-1
EnvProperties.java
src/main/java/com/dsk/flink/dsc/utils/EnvProperties.java
+10
-0
EtlUtils.java
src/main/java/com/dsk/flink/dsc/utils/EtlUtils.java
+6
-0
No files found.
src/main/java/com/dsk/flink/dsc/sync/SyncCustomerDataSource.java
View file @
4fc840af
...
...
@@ -51,7 +51,7 @@ public class SyncCustomerDataSource {
envProps
.
put
(
"providerImpl"
,
JdbcConnectionProviderFactory
.
HikariDataSourceJdbcConnectionProvider
.
class
.
getName
());
//System.out.println("读取到的配置文件:-> " + envProps.toString());
//TODO 到时需要改这里,改成正式的消费组
FlinkKafkaConsumer
<
String
>
kafkaConsumer
=
new
FlinkKafkaConsumer
<
String
>(
envProps
.
getKafka_topic
(),
new
SimpleStringSchema
(),
EtlUtils
.
getKafkaConfig
(
envProps
.
getKafka_brokers
(),
envProps
.
getKafka_topic
().
concat
(
"-group"
),
envProps
.
getKafka_username
(),
envProps
.
getKafka_password
()));
FlinkKafkaConsumer
<
String
>
kafkaConsumer
=
new
FlinkKafkaConsumer
<
String
>(
envProps
.
getKafka_topic
(),
new
SimpleStringSchema
(),
EtlUtils
.
getKafkaConfig
(
envProps
.
getKafka_brokers
(),
EtlUtils
.
getKafkaGroup
(
envProps
),
envProps
.
getKafka_username
(),
envProps
.
getKafka_password
()));
//System.out.println(envProps.getKafka_topic());
//偏移量
if
(
StrUtil
.
isNotBlank
(
offsetTimestamp
))
{
...
...
src/main/java/com/dsk/flink/dsc/utils/EnvProperties.java
View file @
4fc840af
...
...
@@ -8,6 +8,8 @@ import java.util.Properties;
*/
public
class
EnvProperties
extends
Properties
{
String
env
;
// #连接
String
db_url
;
...
...
@@ -111,6 +113,14 @@ public class EnvProperties extends Properties {
String
solr_urls
;
String
solr_zk_hosts
;
public
String
getEnv
()
{
return
env
==
null
?
this
.
getProperty
(
"env"
)
:
env
;
}
public
void
setEnv
(
String
env
)
{
this
.
env
=
env
;
}
public
String
getDb_url
()
{
return
db_url
==
null
?
this
.
getProperty
(
"db_url"
)
:
db_url
;
}
...
...
src/main/java/com/dsk/flink/dsc/utils/EtlUtils.java
View file @
4fc840af
package
com
.
dsk
.
flink
.
dsc
.
utils
;
import
cn.hutool.core.util.StrUtil
;
import
java.util.Properties
;
public
class
EtlUtils
{
...
...
@@ -31,6 +33,10 @@ public class EtlUtils {
return
String
.
format
(
saslJaasConfig
,
username
,
password
);
}
public
static
String
getKafkaGroup
(
EnvProperties
envProps
){
return
envProps
.
getKafka_topic
().
concat
(
StrUtil
.
isNotBlank
(
envProps
.
getEnv
())
?
envProps
.
getEnv
()
:
""
).
concat
(
"-group"
);
}
public
static
void
main
(
String
[]
args
)
{
System
.
out
.
println
(
getSaslJaasConfig
(
"szx"
,
"123456"
));
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment